Draft SOW.pdf

STATEMENT OF WORK (SOW)

Scientific Information Management and Literature‐based Evaluations for the National Institute of Environmental Health Sciences

November 13, 2023

1 General Information

1.1 General

This is a non‐personal service(s) contract under which the personnel rendering the service(s) are not subject, either by the contract’s terms or by the manner of its administration, to the supervision and control usually prevailing in relationships between the Government and its employees. The purpose of this contract is to provide all services and supplies necessary to conduct scientific information management and literature‐based evaluations in support of the National Institute of Environmental Health Sciences (NIEHS), Division of Translational Toxicology

(DTT).

1.2 Description of Services

Independently, and not as an agent of the Government, the Contractor shall furnish all the necessary services, qualified personnel, procedures, materials, equipment, and facilities not otherwise provided by the Government as needed to perform the objectives set forth below. In general, studies shall be conducted according to the requirements described in a research outline (see below), which includes project‐specific activities, milestones, and deliverables.

1.3 Background and Scope

DTT is an intramural division of NIEHS, National Institutes of Health that uses traditional and cutting‐edge research approaches to better understand how factors in our environment may impact human health. Such approaches include using scientific expertise to identify, collect, analyze, and interpret diverse types of information relevant to the effects of environmental exposures on human health. Information from the fields of toxicology, epidemiology, chemistry, data science, informatics, pathology, and other disciplines is used in the design of research programs; in literature‐based health assessments; and in the development of novel tools and informatics approaches.

The objective of this contract is to provide scientific and technical expertise and support for scientific information management and literature‐based evaluations related to the evaluation and reporting of effects of environmental substances and other issues that may impact public health. Nine specific requirement areas under this contract to support DTT within NIEHS include: (1) systematic evidence mapping and scoping reviews of environmental health literature, (2) systematic review and evidence synthesis, (3) report development, (4) document preparation and publication, (5) coordination and management of external peer review, (6) logistical management for advisory group meetings and workshops, (7) provision of scientific or technical consultants with expertise in relevant scientific disciplines, (8) development and implementation of novel tools and informatics approaches, and (9) program management.

These requirement areas are managed by several offices and branches within DTT. Additional information about these groups is available on the NIEHS website.

The specific activities related to the nine requirements are described in Section 2. These requirements support four primary research programs within DTT, including (1) exposure‐based research programs, (2) health effects innovation programs, (3) responsive research programs, and (4) strengthening capabilities platforms. Additional information on these research programs can be found on the NIEHS website.

The National Toxicology Program (NTP) is an interagency partnership comprised of NIEHS (within the National Institutes of Health), the National Institute for Occupational Safety and Health (NIOSH; within the Centers for Disease Control and Prevention), and the U.S. Food and Drug Administration. NTP is headquartered administratively at NIEHS, and many of the activities performed under this contract will support NTP projects and research programs.

The activities carried out by this contract will address the aforementioned requirements for scientific information management.

1.4 Overview of Required Capabilities

The Contractor shall conduct activities supported by the following Capability Areas described in more detail in Section 2 – Required Capability Areas:

Systematic Evidence Mapping and Scoping Reviews of Environmental Health Literature

Systematic Review and Evidence Synthesis

Report Development

Document Preparation and Publication

Coordination and Management of External Peer Review

Logistical Management for Advisory Group Meetings and Workshops

Provision of Scientific or Technical Consultants with Expertise in Relevant Scientific Disciplines

Development and Implementation of Novel Tools and Informatics Approaches

Program Management

1.5 Policies and Procedures

For scientific information management and literature‐based evaluations, the Contractor and applicable subcontractor(s) shall comply with the below guidelines:

a. Handbook for Conducting Systematic Reviews for Health Effects Evaluations

Available at: https://ntp.niehs.nih.gov/go/ohathandbook.

b. Handbook for Preparing Report on Carcinogens Monographs

Available at: https://ntp.niehs.nih.gov/go/rochandbook.

1.6 Reports and Deliverables

Information on specific reporting requirements is presented below. All deliverables shall be electronic and PC and MAC compatible unless otherwise specified by the Government. Each final document or report shall be submitted electronically in MS Word and/or PDF formats to the Contracting Officer’s Representative (COR) for review. Electronic deliverables must be compliant with Section 508 regarding accessibility (see Section 1.6.4). In some cases, there may be accommodations for which the Government will provide information annually.

In addition to the reports mentioned herein, any incidence or unforeseeable occurrence not listed here, including personnel changes which may impact activities under the contract, shall be immediately reported to the Government.

DELIVERABLE REFERENCE DUE/FREQUENCY SUBMIT TO

Quality Management Plan

Section 1.6.1.1 Revise as needed or when major changes occur

COR, CO, and CS

Monthly Progress Report

Section 1.6.1.2 Due once per month COR, CO, and CS

Monthly Budget Summary Report

Section 1.6.1.3 Due once per month as an attachment with the monthly invoices

COR, CO, and CS

Annual Report Section 1.6.1.4 Due 4 weeks from the last day of each contract year

COR, CO, and CS

Technical Work Plan and Estimated Budget

Sections 3.1 and 8.1

Due 3 weeks from research outline submission. Due dates may vary based on the individual research outline assigned.

COR, CO, and CS

COR = Contracting Officer’s Representative; CO = Contracting Officer; CS = Contract Specialist.

1.6.1 Periodic Reports

1.6.1.1 Quality Management Plan

A Quality Management Plan (QMP) shall be submitted before performance of work under the contract and updated versions submitted at regular intervals (e.g., biannually).

1.6.1.2 Monthly Progress Report

Once per month, the COR shall receive a summary of the work currently in progress. This report shall include details on the status of all active work, current costs for active work, cumulative costs for the contract (including direct and administrative costs), any problems encountered, and remedial actions taken to resolve those problems. The monthly progress report shall also include a summary of the work performed by any subcontractor and/or ad hoc consultant, including their deliverables, hours spent, and cost. This summary shall also include a description of how the subcontractor/consultant work aided the Contractor in completing or nearing completion of a specific project.

1.6.1.3 Budget Summary Report

A summary report shall be submitted monthly that includes budget information for each activity separately undertaken and for the contract as a whole. The report must include each activity’s budget, total expenditures on the activity for the month, summaries of current contract year and cumulative expenditures, and projected expenses to complete each activity.

Budget summary reports shall be submitted monthly to the COR as MS Excel spreadsheets (or in other spreadsheet software by agreement) and shall be sortable as required by the COR.

Budget summary reports shall be included as an attachment with the monthly invoices.

1.6.1.4 Annual Report

An annual report shall be submitted within 4 weeks of the last day of each contract year. Each annual report shall detail all significant outcomes resulting from the requirements as described within the SOW and contract, including a breakdown of costs by research activity incurred to date, any problems encountered and remedial actions taken to resolve the problem, and any deviations from the SOW and/or contract.

1.6.2 Research Products

For each activity performed under the contract, research products shall be generated by the Contractor and submitted to the COR according to the milestones and deliverables schedule included in the research outline. The type and number of product(s) may vary depending on the nature of the activity.

1.6.3 IT Security Plan and Other IT Deliverables

Refer to Appendix A for specific requirements related to information technology (IT) Security.

1.6.4 HHS Section 508 and Accessibility Standards

Section 508 of the Rehabilitation Act, as amended by the Workforce Investment Act of 1998 (P.L. 105‐220) requires that when Federal agencies develop, procure, maintain, or use information and communication technology (ICT), it shall be accessible to people with disabilities. Federal employees and members of the public who have disabilities must have access to, and use of, information and data that is comparable to people without disabilities.

All products, platforms and services delivered as part of this work statement that are ICT, or contain ICT, must conform to the Revised 508 Standards, which are located at 36 C.F.R. §

1194.1 & Apps. A, B, & C, and available at https://www.access‐board.gov/ict/.

All requirements are applicable to support services and documentation deliverables. All functional performance criteria apply when using an alternative design or technology that achieves substantially equivalent or greater accessibility and usability by individuals with disabilities, than would be provided by conformance to one or more of the requirements in Chapters 4‐6 of the Revised 508 Standards, or when Chapters 4‐6 do not address one or more functions of ICT.

For each proposed product or platform, a fully completed Accessibility Conformance Report (ACR) using the Voluntary Product Accessibility Template (VPAT) (https://www.itic.org/policy/accessibility/vpat) or HHS conformance checklist(s) (https://www.hhs.gov/web/section‐508/accessibility‐checklists/index.html) must be submitted.

Prior to acceptance of deliverables, the offeror must demonstrate conformance to the HHS Section 508 requirements via HHS Section 508 checklist(s) (https://www.hhs.gov/web/section‐ 508/accessibility‐checklists/index.html). The Government reserves the right to perform testing on required ICT items to validate the offeror’s Section 508 conformance claims. If the Government determines that Section 508 conformance does not meet the HHS Section 508 requirements, the Government shall, at its option, require the offeror to remediate the item, at no additional cost to the Government, to align with the HHS Section 508 conformance requirements prior to acceptance.

References:

Section 508 Standards: https://www.access‐board.gov/ict/

HHS Policy on Section 508 Compliance and Accessibility of Information and Communications Technology (ICT): https://www.hhs.gov/web/governance/digital‐ strategy/it‐policy‐archive/hhs‐policy‐section‐508‐compliance‐accessibility‐information‐ communications‐technology.html

HHS Accessibility and Section 508 Conformance Checklists:

https://www.hhs.gov/web/section‐508/accessibility‐checklists/index.html

1.7 Post Award Conference/Periodic Progress Meetings

The Contractor shall attend any post award conferences convened by the contracting activity or contract administration office in accordance with FAR Subpart 42.5. The Contracting Officer (CO), COR, and other Government personnel, as appropriate, may meet periodically with the Contractor to review the Contractor's performance. At these meetings the CO will apprise the Contractor of how the Government views the Contractor's performance and the Contractor shall apprise the Government of problems, if any, being experienced. Appropriate action shall be taken to resolve outstanding issues. These meetings shall be at no additional cost to the Government.

1.7.1 Progress Meetings

The Contractor shall attend, participate in, and furnish input to scheduled and unscheduled meetings, conferences, and briefings that relate to the functions and services herein as required by the Government to provide effective communication and impart necessary information. The Principal Investigator (PI) or designated representative shall attend meetings as requested by the Government. Meeting attendees shall at times include Contractor managerial, supervisory, and other personnel knowledgeable of the subject matter. Meetings may start or end outside of regular duty hours.

1.8 Contracting Officer’s Representative

As determined by the CO, a COR will be appointed and identified by letter of designation, a copy of which will be provided to the Contractor by the CO. The designation letter states the responsibilities and limitations of the COR.

1.9 Personnel

This contract requires a high degree of collaboration among (or by) Contractor staff. Each requirement described in this SOW shall be performed by an appropriate mix of professional and technical staff. Personnel assigned to this contract, including subcontractors and/or consultants, shall demonstrate that there are no conflicts of interest prior to performing work under the contract. Personnel assigned to this contract with access to federal IT resources may be required to pass a security investigation (see Section 4).

The Contractor shall ensure that all Contractor personnel are adequately trained, possess the required credentials, and are fully qualified and skilled to perform the activities outlined in this SOW. The Contractor shall be responsible for all costs associated with training and necessary travel to ensure that staff maintain required certifications and proficiency to perform assigned duties. Contract personnel shall have the expertise to contribute to problem formulation and research design as required.

Contractor staff designated as key personnel must be approved by the COR. Contractor staff may need to be available after normal working hours and on weekends. The Contractor shall employ and/or have access to staff with broad professional and technical expertise in environmental health sciences necessary to conduct evaluations of complex scientific literature and fulfill the activities of the SOW.

1.9.1 Professional Personnel

The PI and the Program Manager (PM) are both considered key personnel. The PI and the PM shall have appropriate training, experience, and scientific expertise and shall be familiar with the management and coordination of multidisciplinary projects relevant to the requirements for scientific information management and literature‐based evaluations described in the SOW.

Senior toxicologists, senior epidemiologists, staff toxicologists, and staff epidemiologists are considered professional/critical staff due to the nature of the data used and generated by DTT, and these staff shall be able to use independent judgment based upon their scientific expertise.

The daily interaction and constant coordination of efforts needed among these disciplines requires that they be organizationally together.

Senior toxicologists shall have appropriate education, training, and experience in (1) the conduct and/or evaluation of studies in toxicology, carcinogenicity, reproductive and developmental biology, or a related field; (2) evaluating study quality, interpreting evidence from individual studies, and integrating evidence across a collection of relevant studies; (3) extracting, visualizing, and/or analyzing data; and (4) working collaboratively and preparing scientific reports.

Senior epidemiologists shall have appropriate education, training, and experience in (1) the conduct and/or evaluation of environmental and occupational observational epidemiology studies in health‐related fields; (2) evaluating study quality, interpreting evidence from individual studies, and integrating evidence across a collection of relevant studies; (3) extracting, visualizing, and/or analyzing data; and (4) working collaboratively and preparing scientific reports.

Staff toxicologists and staff epidemiologists shall have appropriate training and experience in conducting or preparing scientific reviews of toxicological or observational epidemiological studies, including extracting data from studies; evaluating study quality; interpreting and summarizing study findings; and extracting, visualizing, and/or analyzing data.

1.9.2 Technical Personnel

Scientific expertise may be required in chemistry, toxicology, environmental and occupational epidemiology, cancer epidemiology, statistics, mechanisms of effect, in vitro analyses, computational biology, data science, informatics, environmental exposures (e.g., occupational hygienists, exposure science), read‐across, genotoxicity, disposition and metabolism, carcinogenicity, immunotoxicology, reproduction, development, or other areas relevant to the project. In addition, supporting staff are required for the following areas: scientific writing and editing, data visualization, and meeting logistics. The level of effort for each discipline, at any time, may vary from year to year based on the complexity, nature, and extent of the assigned work. When experts who are not part of the Contractor’s staff are needed, the Contractor shall have procedures in place to ensure the needed expertise can be obtained and provide the COR within one week a plan for obtaining that expertise.

1.10 Data Rights, Patents, and Copyrights

The Government has unlimited rights to all documents and materials produced under this contract. All documents and materials, including the source codes of software, produced under this contract shall be Government owned and are the property of the Government with all rights and privileges of ownership and copyright belonging exclusively to the Government.

These documents and materials may not be used or sold by the Contractor without written permission from the CO. All materials supplied to the Government shall be the sole property of the Government and may not be used for any other purpose. This right does not abrogate any other Government rights.

1.10.1 Source Documentation

All original source documentation is the property of NIEHS. Data summaries and other relevant information generated during the performance of this contract shall be submitted to the COR in the form of status reports and agent specific reports as specified in the Deliverables section above. In addition, annual reports shall be submitted for all activities as specified in the Deliverables section below. To the extent possible, the Government will retain unrestricted access and rights to all data and technical information generated under the contract.

1.11 Non‐Disclosure Requirements

Performance under this contract may require the Contractor to access data and information proprietary to a Governmental agency, another Government Contractor, or of such nature that its dissemination or use other than as specified in this work statement would be adverse to the interests of the Government or others. The Contractor and Contractor’s employees (including Subcontractors) shall not divulge or release data or information developed or obtained under performance of this contract, except to authorized Government personnel or upon written approval of the CO. The Contractor shall not use, disclose, or reproduce proprietary data, which bears a restrictive legend, other than as specified in this contract. All documentation showing individual names or other personal information shall be controlled and protected under the provisions of the Privacy Act of 1974, Public Law 93‐579, 5 United States Code (U.S.C.) Section 552a.

1.12 Replication, Dissemination, or use of the Results

It is anticipated that the results of the research conducted under the contract shall be published in the open, peer‐reviewed scientific literature and/or in a NTP or NIEHS report series. Specific details concerning authorship and technical issues surrounding the publication of the research shall be discussed and agreed upon by the PI and the DTT COR on a case‐by‐case basis.

Because premature release of information developed under the contract could be harmful to the public, the Contractor and the Government agree that it is important to consider issues related to the timing of the release of research results when submitting information for publication. Regarding public disclosures of the research to be carried out pursuant to this contract, the Contractor agrees to submit all proposed manuscripts, abstracts, or reports to the COR at least thirty (30) days in advance of submission for publication. Within this thirty‐day period the COR shall inform the Contractor if there are serious concerns raised by the timing of said publication. Upon receipt of a notification that the Government objects to the publication of said information, the Contractor agrees to withhold submission of said publication while the author and the COR discuss the issues of concern. The Contractor agrees to give good faith consideration to all such issues and further agrees that it will not knowingly submit for publication any article, such as a manuscript, abstract, or other item, that would be harmful to the public welfare. The final decision on whether to submit an article for publication shall rest with the DTT COR after the procedures set forth above are followed. Similar procedures shall apply for scientific presentations and meeting abstracts.

1.13 Protection of Government and Contract Information

The Contractor shall not cite any information (e.g., contract information, pictures, locations, etc.) obtained through this contract on any hard copy or digital marketing tools to include its company website.

1.14 Non‐Disclosure Statements

The Contractor shall provide signed non‐disclosure agreements to the Government no later than five (5) prior to commencement of work under the contract. Disclosure of information by Contractor personnel may result in removal of Contractor personnel from performance under this contract.

1.15 Safety Laws and Regulations

The Contractor shall comply with FAR 52.236‐13, Accident Prevention (November 1991).

In addition to other requirements for accident prevention specified in this contract, the Contractor shall comply with all Federal and State requirements governing safety and health protection.

The Contractor shall indicate its plan for compliance considering conditions of the work described by the design, technical proposal, operational activities of the Government or others in the area of the work, identification of the coordinator, and implementation of all other accident prevention provisions of this contract.

The Contractor shall complete and provide safety notifications as required by national, state, and local regulatory bodies, and will provide documented proof to the CO via the COR, no later than 10 days that such notifications were made.

2 Required Capability Areas

2.1 General Requirements

The Contractor shall provide services, qualified personnel, equipment, and facilities needed to support scientific information management and preparation of documents and products under this contract.

2.2 Systematic Evidence Mapping and Scoping Reviews of Environmental Health Literature

The Contractor shall provide staff with expertise in literature searching, literature evaluations, and the management of systematic evidence mapping and scoping reviews. Domain‐specific expertise for this capability area shall include toxicology and subdisciplines (e.g., reproductive toxicology, cancer), epidemiology, chemical or biological properties, human exposure, mechanistic data, and informatics. Staff shall have access to a variety of specialized databases and information resources related to environmental health.

Most activities conducted under this capability area will require support as outlined in Section 2.2.1; however, in some cases, Contractor support for a more limited scoping review will be required to address a given research question or develop a research concept. For example, brief or targeted scoping reviews may be part of problem formulation to refine the exposure, outcomes, or animal models of interest.

2.2.1 Systematic Evidence Mapping

The Contractor shall provide staff with the appropriate expertise to manage and support the development of systematic evidence maps (SEMs) including literature searching, literature screening, data extraction, data categorization, and data visualization. All systematic evidence mapping activities shall be conducted according to the first three steps outlined in the Handbook for Conducting Systematic Reviews for Health Effects Evaluations1 (problem formulation, search and selection of studies for inclusion, data extraction) with procedures adjusted to address a given topic.

2.2.1.1 Literature Searching

Literature searching to support systematic evidence mapping shall be conducted with the rigor required for systematic review. To identify relevant scientific literature for a SEM, the Contractor shall develop focused and/or comprehensive literature or information search strategies in consultation with the COR. Literature or information searches shall be conducted using multiple scientific literature databases and other relevant sources (e.g., websites, exposure and authoritative sources) and literature or data retrieved from these resources shall be organized using reference management software, preferably EndNote™. The Contractor shall also provide a mechanism for storage and management of all electronic data or literature files (e.g., file management software such as SharePoint/OneDrive).

As needed, the Contractor shall conduct searches of relevant literature databases for information on a particular agent (chemical, physical, biological) or stressor and reported associated health effects. In addition, information on use, occurrence, human exposure, and guidelines and regulations for substances under evaluation may also be collected.

Occasionally, searches of databases not available through the NIEHS or NIH library may be needed, and this will be determined on a case‐by‐case basis after discussions with the COR. In addition, translation of studies published in foreign journals may be needed, and this will be

1 Available at: https://ntp.niehs.nih.gov/go/ohathandbook.

determined on a case‐by‐case basis after discussions with the COR. The Contractor shall arrange for translation services as necessary.

The Contractor shall make accessible to Government staff electronic, word‐searchable, portable document format (PDF) files for all references cited in all drafts of scoping documents (see Section 0). As directed, the Contractor shall make the references electronically available, either in a transmittable EndNote file or via a shared online location (i.e., SharePoint, Teams, OneDrive), depending upon instructions from the COR. The Contractor shall provide a mechanism for file sharing of materials and references among designated Government staff or consultants (e.g., using project management software).

The Contractor shall arrange for and/or acquire any needed information resources or information management software following discussion and approval by the COR.

2.2.1.2 Literature Screening

Literature screening to support systematic evidence mapping shall be conducted with the rigor required for systematic review. The Contractor shall use web‐based, secure literature screening software appropriate for supporting the transparency and reproducibility of systematic review methods for all literature screening activities. The Government will provide access to and/or reimbursement for all required software. Software examples include, but are not limited to:

DistillerSR [https://www.distillersr.com/products/distillersr‐systematic‐review‐software]

Health Assessment Workspace Collaborative (HAWC) [https://hawcproject.org/]

SWIFT Active Screener [https://www.sciome.com/swift‐activescreener/]

Table Builder [https://ehp.niehs.nih.gov/doi/full/10.1289/EHP4224]

The Contractor shall screen relevant literature using state‐of‐the‐art approaches including text mining, machine learning, artificial intelligence (AI)‐enabled, and/or manual screening; select literature using specific inclusion/exclusion criteria; and track the output of literature screening in detail for display or data storage. Studies identified as relevant will be broadly categorized into an appropriate evidence stream or selected scientific topic (e.g., human or experimental animal studies, exposure data, other relevant data including in vitro or mechanistic) as defined by the scope of the evaluation and input from the Government.

2.2.1.3 Scientific Data Extraction, Categorization, Management, and Visualization The Contractor shall extract specified data from relevant scientific studies, as defined by the Government, into databases (e.g., HAWC or Table Builder) using specific web‐based forms which are easily exportable to Excel and Word tables. Data shall be extracted using tiered, hierarchical methods that support interactive evidence mapping.

The Contractor shall prepare tabular and graphical displays of extracted data and information using templates and tools specified by the Government. The Contractor shall arrange to have access to software for visualizing data (e.g., Tableau) and drawing chemical structures and/or biological pathways (e.g., ChemSpider, Biocarta, or Ingenuity® Pathway Analysis). The Contractor shall be able to provide outputs from data visualization software in a format specified by the Government (e.g., as a PDF or in an editable format). The Contractor shall arrange for and/or acquire any needed subscriptions or tools upon approval from the CO.

2.2.2 Scoping Reviews

The Contractor shall provide support for developing scoping reviews as needed for a given research question or research concept. For example, brief or targeted scoping reviews may be part of problem formulation to refine the exposures, outcomes or animal models of interest.

Scoping reviews may be used to develop a literature search strategy and determine the “scope” or extent of the literature necessary for addressing a research question. Scoping reviews may also be conducted more formally using systematic review‐level transparency and documentation to identify and summarize relevant studies on a given topic. Additional details on requirements for literature searching, literature screening, data extraction, and data visualization are available above in Section 2.2.1.

2.2.3 Development and Publication of Systematic Evidence Mapping and Scoping Documents

In coordination with the Government, the Contractor shall develop systematic evidence mapping and scoping documents for publication, including, but not limited to, protocols, peer‐ reviewed manuscripts, and NIEHS Reports (see Section 2.4). The Contractor shall provide support for compiling documents based on Government guidance, reviewing documents for grammatical correctness, and performing technical review for accuracy and internal consistency to ensure report quality. All documents prepared for publication as Government reports shall be prepared in a predefined template and writing style as described in Section 2.5.

2.2.4 Conflict of Interest Screening

The Contractor shall identify industries and companies involved in the manufacture, distribution, and/or use of specific substances or issues that are the topic of literature‐analysis activities or updates for current similar reports. The Government would use this information to identify potentially affected parties for inclusion in conflict‐of‐interest (COI) screening of employees of the Contractor, subcontractors, consultants, or other interested parties.

2.3 Systematic Review and Evidence Synthesis

The Contractor shall provide staff with the appropriate expertise to manage and support systematic reviews and evidence synthesis for health hazard assessments, including literature searching, screening, data extraction and visualization, study quality assessment, data/evidence interpretation, and scientific evidence integration. Domain‐specific expertise for systematic reviews and evidence synthesis shall include toxicology and subdisciplines (e.g., reproductive toxicology, cancer), environmental and occupational epidemiology, human exposure (i.e., occupational hygiene), chemistry (including absorption, distribution, metabolism, and excretion), mechanistic data, and informatics. All systematic review activities to support health hazard assessments shall be conducted according to procedures outlined in the Handbook for Conducting Systematic Reviews for Health Effects Evaluations2 and the Handbook for Preparing Report on Carcinogens Monographs3.

2 Available at: https://ntp.niehs.nih.gov/go/ohathandbook.

3 Available at: https://ntp.niehs.nih.gov/go/rochandbook.

2.3.1 Literature Search and Screening

Literature searching and screening to support systematic review and evidence synthesis shall be conducted with the rigor required for systematic review. Additional details on requirements for literature searching and literature screening are available above in Section 2.2.1.

2.3.2 Scientific Data Extraction, Categorization, Management, and Visualization The Contractor shall extract specified data from relevant scientific studies, as defined by the Government, into databases (e.g., HAWC or Table Builder) using specific web‐based forms which are easily exportable to Excel and Word tables. Details on methods for data extraction are provided in the respective handbooks referenced above. Note that these approaches are similar to those outlined in Section 2.2.1, but likely to capture study data in more detail than described above for systematic evidence mapping.

The Contractor shall prepare tabular and graphical displays of extracted data and information using templates, guidance, and tools specified by the Government. The Contractor shall arrange to have access to software for visualizing data (e.g., Tableau) and drawing chemical structures and/or biological pathways (e.g., ChemSpider, Biocarta, or Ingenuity® Pathway Analysis). The Contractor shall be able to provide outputs from data visualization software in a format specified by the Government (e.g., as a PDF or in an editable format). The Contractor shall arrange for and/or acquire any needed subscriptions or tools upon approval from the CO.

2.3.3 Study Quality Assessment, Data Analysis, Data Interpretation, and Scientific Evidence Integration

The Contractor shall review published scientific studies, summarize relevant information, and assist with assessment of study quality including, but not limited to, risk‐of‐bias assessment.

The Contractor shall use scientific knowledge and independent judgment to assist with data analysis (e.g., means or meta‐analyses), data interpretation, and evidence integration. The Contractor shall utilize study quality assessment methods or frameworks and structured forms specified by the Government to facilitate assessment of study quality. The output from the study quality assessment shall be presented in textual, tabular, or graphic formats, with documentation of scientific judgement and the basis of decisions as required.

The Contractor shall also provide statistical and graphing support for data analysis. Examples of this support include, but are not limited to, conducting meta‐analyses or preparing stratified forest plots (e.g., using Stata or R) of human epidemiological studies, conducting trend‐ and pairwise analyses of data from animal studies, and evaluating mechanistic data.

The Contractor shall review the available scientific evidence on a given health effect and use scientific judgment to draft text that critically evaluates and interprets the scientific findings of individual studies and/or integrates the scientific evidence across studies for multiple evidence streams (e.g., human, experimental animal, and mechanistic studies). Evidence evaluation includes assessment of the impact of bias on the study findings.

If requested, the Contractor shall also evaluate and summarize exposure and other related information that may support a health hazard assessment.

2.3.4 Development and Publication of Systematic Review Documents Research findings from systematic reviews are published in the NTP Monograph report series (noncancer health effects), the Report on Carcinogens (RoC) Monograph report series (cancer health effects), or as peer‐reviewed journal publications. As described in Section 2.5, the Contractor shall provide support for compiling documents based on Government guidance, preparing reports in a predefined template and writing style, reviewing documents for grammatical correctness, and performing technical review for accuracy and internal consistency to ensure report quality.

In addition to publication of RoC Monographs, the Contractor shall provide support for publication of the Report on Carcinogens (RoC). The RoC is a congressionally mandated, science‐based, public health report that provides a cumulative list of known or reasonably anticipated human carcinogens; the RoC is updated with new listings every few years.

Preparation of each new edition requires updating the information in all or selected profiles, including supporting information, exposure‐related information, regulatory information, and other information for all or selected profiles. These activities include conducting information searches, extracting and summarizing relevant information, and integrating the information into the existing profiles. The Contractor shall also provide support for creating individual files for each profile and compiling and publishing the RoC according to Government guidance.

2.3.5 Conflict of Interest Screening

The Contractor shall identify industries and companies involved in the manufacture, distribution, and/or use of specific substances or issues that are the topic of literature‐analysis activities or updates for current similar reports. The Government would use this information to identify potentially affected parties for inclusion in COI screening of employees of the Contractor, subcontractors, consultants, or other interested parties.

2.4 Report Development

Research findings from DTT, NIEHS, and NTP studies are published in manuscripts and several different report series, including NTP Technical Reports, NTP Toxicity Reports, NTP Developmental and Reproductive Toxicity (DART) Reports, NTP Immunotoxicity Reports, NTP Research Reports, NIEHS Reports, and other potential report series. Preparation of these documents uses a series of drafts to review and refine data presentation and interpretation, and these reports go through the NIEHS/DTT internal review process continually throughout document preparation. The Contractor shall provide support for compiling documents based on Government guidance and laboratory study reports, preparing reports in a predefined template and writing style, reviewing documents for grammatical correctness, and performing technical review for accuracy and internal consistency to ensure report quality. In addition, report development support shall be provided for documents post‐peer review, including reviewing responses to reviewer comments for accuracy and completeness, revising documents based on peer review comments, and incorporating findings from third‐party quality assessment review.

Additional details on requirements for document preparation and publication can be found in Section 2.5. Example publications can be found here:

https://ntp.niehs.nih.gov/publications/index.html.

Support for report development requires access to internal NIEHS databases, as described below in Section 4.

2.5 Document Preparation and Publication

Documents prepared under this contract may be presented as a mix of graphical, tabular, and textual formats. The Contractor shall verify the scientific and technical accuracy of information or data included in all documents whether written by Government staff, the Contractor, or consultants.

The Contractor shall provide administrative and technical support for the preparation and publication of documents, as needed, including writing text using a concise and technically correct format understandable by a wide audience, and assimilating information into reports according to specified style templates and, as required, 508‐compliant PDF documents. General administrative and technical skills needed by the Contractor to support this requirement include experience with Microsoft Office, Adobe Acrobat, Section 508‐compliance, and records management.

Using a publication style guide provided by the Government, the Contractor shall provide support for formatting documents, copyediting and technical editing all report types, formatting references using software such as EndNote™, and readying reports for publication according to a predefined template format that was previously developed by eXtyles for DTT Government reports. Final documents shall be available in electronic formats and be prepared as 508‐compliant PDF documents (see Section 1.6.4). Final 508‐compliant documents will be posted to the NTP and/or NIEHS public websites and will also be archived in a shared online location (i.e., SharePoint, Teams, OneDrive). Actual posting of final documents to Government websites will be conducted by a NIEHS/DTT web support contractor and is not required as part of this SOW.

The Contractor shall provide support for the NIEHS/DTT electronic publishing workflow, including preparation of template and file conversions for Web publishing of RoC and NTP monographs, NTP reports, NIEHS reports, and Government documents as identified by the COR.

Government‐specified software (e.g., eXtyles) shall be utilized to convert final document files to XML for subsequent upload and publication on the National Center for Biotechnology Information (NCBI) Bookshelf website (https://www.ncbi.nlm.nih.gov/books/). The Contractor shall also submit citation information for all final documents to NCBI for indexing within PubMed (https://pubmed.ncbi.nlm.nih.gov/).

Occasionally, the Contractor, as required by the COR, shall provide support in obtaining document covers and related graphics. The Contractor shall interface with the NIEHS arts/graphics contractor for their preparation as instructed by the COR.

In addition to preparing new documents, the Contractor shall provide support for updating existing documents, including reviewing information and content, re‐formatting, and preparing documents for electronic publication (e.g., NTP and NIEHS public websites, NCBI Bookshelf/PubMed).

The Contractor shall provide printing services for draft or final documents or products as required and approved by the COR.

2.6 Coordination and Management of External Peer Review

RoC and NTP monographs and NTP and NIEHS reports are published in NCBI’s Bookshelf (as described in Section 2.5) and are required to go through external peer review prior to publication. Peer review may occur via individual letter review (like a peer‐reviewed manuscript) or via expert panel review at a public meeting. The Contractor shall provide support for external peer review of NIEHS and NTP documents, including identifying experts, developing COI packages, screening potential experts, corresponding with reviewers, preparing review documents, and post‐peer review activities. Upon approval from the COR, the Contractor shall obtain and manage subcontracts with identified peer reviewers and pay them for their services.

Experts on a specific topic shall be identified utilizing the online NTP Expert Directory (accessible via the NIH network) as well as publicly available resources. COI packages (including lists of potentially affected companies) shall be prepared based on a given report’s topic and all identified experts shall be screened for potential real or perceived COI. The Contractor shall screen and/or assist the Government in obtaining COI information from scientific experts prior to confirmation of service. Procedures developed around expert selection and COI screening should demonstrate how to handle personally identifiable information (PII). In addition, the Contractor shall be familiar with the requirements around influential scientific information (ISI) and highly influential scientific information (HISI) in accordance with the Office of Management and Budget’s Information Quality Bulletin for Peer Review (see Section 7, Applicable References).

2.7 Logistical Management for Advisory Group Meetings and Workshops Information management, literature analysis, and report development activities within NIEHS/DTT may require internal and/or external meetings or events that are remote (web‐ based, e.g., webcast, Zoom, or similar), in‐person, or a hybrid of the two, and can vary in size (e.g., 4‐6 in‐person or up to 200 attendees at a webinar). The types of events can vary and may include advisory group meetings, peer reviews, symposia, workshops, focus/information groups, and seminars. The events will vary in length, generally from one‐half to two days, and will usually be held or coordinated from the NIEHS campus or in the Research Triangle Park, NC or Washington, DC areas, although occasionally they may be held at another domestic or international location. The number of experts required for an event will vary, ranging, for example, from 3‐4 speakers for a webinar or symposium up to 10‐20 experts for an advisory board meeting. The Government will specify whether the meeting is closed or open to the public and other specific details.

The Contractor shall provide administrative, technical, and logistical support for the planning and execution of these meetings including identification of an appropriate venue, although use of Government space is priority. As requested, the Contractor shall assist the Government in obtaining information from participants for COI screening. The Contractor shall handle travel arrangements for participants, subject to applicable Government regulations. Prior to confirming any arrangements for meetings, the Contractor shall consult with and receive approval from the COR. The Contractor shall provide support for managing or preparing materials for the meeting and drafting any needed documents, minutes/reports, notes, or summaries of major points from reports, minutes, or public comments. The Contractor shall also provide support for preparing information for meeting website(s), distributing materials, and handling communications with targeted individuals. Upon approval from the COR, the Contractor shall obtain and manage subcontracts with identified participants and pay them for their services.

Examples of support for advisory group meetings include support for the NTP Board of Scientific Counselors (BSC), the DTT BSC, the Scientific Advisory Committee on Alternative Toxicological Methods (SACATM), expert panels, and others. These advisory groups are federally chartered and provide rigorous scientific peer review and advice on programs and activities. Additional information on these advisory groups can be found here: https://ntp.niehs.nih.gov/go/advisory.

All advisory meetings are conducted according to the Federal Advisory Committee Act.

2.8 Provision of Scientific or Technical Consultants with Expertise in Relevant Scientific Disciplines

The Contractor shall develop knowledge and key contacts to assist with identifying appropriate experts including consultants, writers, peer reviewers, and speakers. A consultant may be either a Government or non‐Government expert who provides information on a specific scientific topic. Areas of scientific expertise for these individuals shall include, but not be limited to, toxicology, epidemiology, pathology, exposure assessment, regulatory processes, and other disciplines. The Contractor shall work with the COR regarding requirements (expertise, COI, timelines, etc.), and make recommendations to the COR regarding appropriate experts. Upon approval from the COR, the Contractor shall obtain and manage subcontracts with identified experts and pay for any services. Logistics for individuals serving as members of peer review panels shall be handled as identified in Sections 2.6 and 2.7.

The Contractor shall maintain and regularly update the online NTP Expert Directory hosted on the NIH network.

2.9 Development and Implementation of Novel Tools and Informatics Approaches The Contractor shall provide staff with the appropriate expertise to manage and support the development or utilization of novel tools and informatics approaches for scientific information management. Domain‐specific expertise for novel methods shall primarily include data science and informatics, with familiarity and expertise in toxicology and related fields as needed.

As needed, the Contractor shall facilitate the development or utilization of novel approaches for three key areas of scientific information management: literature‐based health assessments, evaluating research effectiveness, and vocabularies/ontologies. Support for this activity shall include, but may not be limited to:

Development and/or implementation of novel tools and informatics approaches

Evaluation of current approaches, tools, and methodologies

Adaptation of existing approaches to new workflows

Support for tools and approaches already being used in existing workflows

The Contractor shall consult with the COR regarding requirements for these…