Draft SOW_QA.pdf

STATEMENT OF WORK (SOW)

Quality Assessment Support for the National Institute of

Environmental Health Sciences August 9, 2023

1 General Information

1.1 General

This is a non‐personal service R&D contract under which the personnel rendering the service(s) are not subject, either by the contract’s terms or by the manner of its administration, to the supervision and control usually prevailing in relationships between the Government and its employees. The purpose of this contract is to provide all services and supplies necessary to conduct quality assessments in support of the Division of Translational Toxicology (DTT) at the National Institute of Environmental Health Sciences (NIEHS).

1.2 Description of Services

Independently, and not as an agent of the Government, the Contractor shall furnish all the necessary services, qualified personnel, procedures, materials, equipment, and facilities not otherwise provided by the Government as needed to perform the objectives set forth below. In general, assessments shall be conducted according to the requirements described in a research outline (see below), which includes project‐specific activities, milestones, and deliverables.

1.3 Background and Scope

The mission of the NIEHS is to discover how the environment affects people to promote healthier lives. The DTT is an intramural division at NIEHS. DTT scientists use a variety of traditional and cutting‐edge approaches to better understand how factors in our environment may impact our health. DTT offers a unique venue for the testing, research, and analysis of agents of concern, to identify toxic and biological effects, provide information that strengthens the science base in toxicology, and inform decisions by health regulatory and research agencies to safeguard public health. Many of these studies are conducted in support of the interagency National Toxicology Program (NTP), which evaluates the potential toxicity and carcinogenicity of environmental agents that may pose a health hazard to citizens of the United States. The NTP is a virtual interagency partnership comprised of NIEHS (within the National Institutes of Health), the National Institute for Occupational Safety and Health (NIOSH; within the Centers for Disease Control and Prevention (CDC)), and the National Center for Toxicological Research (NCTR; within the U.S. Food and Drug Administration (FDA)).

The DTT, NIEHS, and NTP produce numerous types of reports on their activities. Products include, but are not limited to, technical reports on toxicity and carcinogenicity studies;

literature reviews and assessments of potential hazards to human health; manuscripts; and

DRAFT

data via websites and databases. These products are multidisciplinary compilations, and development of each product may encompass findings from multiple laboratory study reports, ancillary activity reports, and results of peer reviews of data. Development of these products occurs over several years and may involve input from DTT and NIEHS personnel as well as personnel from multiple Contract Research Organizations (CROs), including those supporting the NTP and DTT in analytical chemistry, toxicology, genetic toxicology, absorption, distribution, metabolism, and excretion (ADME) and toxicokinetics (TK), pathology, technical report preparation, archiving, data collection and management, and others.

The purpose of the Quality Assessment Support Contract is to conduct assessments of the accuracy, consistency, and completeness of various aspects of DTT/NIEHS/NTP products and their components, including data, and provide reports that identify areas in the conduct and documentation of studies that merit attention, correction, or improvement. Findings from these assessments will be communicated to DTT staff and management so they may be addressed; this process will aid the DTT in managing the overall quality of DTT or NTP studies, records, data, and reports. This contract is not intended to perform the function of a quality assurance unit in a Good Laboratory Practice environment.

1.4 Overview of Required Capabilities

The Contractor shall provide quality assessment support in the following Capability Areas:

Pathology Tables

NTP and NIEHS Reports

Laboratory Reports

Electronic Information

Manuscripts

Monographs and other literature review products

Combinations of the above

Assessments of similar types of data and reports

1.5 Policies and Procedures

The contractor shall establish and maintain a quality management plan (QMP). A QMP governs quality aspects of all work performed. The QMP shall follow the requirements set forth in the U.S. Environmental Protection Agency (EPA) Requirements for Quality Management Plans (EPA

QA/R‐2).

The quality assessment support contract also ensures the DTT/NTP complies with the NIH Guidelines for Ensuring the Quality of Information Disseminated to the Public (http://aspe.hhs.gov/infoquality/guidelines/NIHinfo2.shtml), by assessing the quality of information that is disseminated in various forms, including information made available on NIEHS and NTP websites.

Audits of toxicology studies conducted by CROs shall be reviewed for compliance in accordance with the Specifications for the Conduct of Toxicity Studies by the Division of Translational Toxicology at NIEHS (https://doi.org/10.22427/NIEHS‐00), referred to hereafter as the DTT Specifications.

Audits of toxicology studies and laboratory reports where the study protocol specifies Good Laboratory Practice (GLP) shall be reviewed for compliance with the GLP Regulations established by the FDA for Non‐clinical Laboratory Studies in the Code of Federal Regulations (CFR) Title 21 (21 CFR part 58: https://www.ecfr.gov/current/title‐21/chapter‐I/subchapter‐ A/part‐58) or as most recently updated.

The Contractor shall document the procedures used to perform each assessment. It is the Contractor’s responsibility to develop, modify, and maintain SOPs as necessary, with appropriate DTT review. Prior to conduct of assessments, the COR shall review appropriate quality assessment support contractor‐generated SOPs.

The Contractor shall prepare a report for each assessment assigned and performed. In general, each report shall contain a detailed narrative of the procedures utilized, an inventory of records and documents used, a detailed tabulation of findings by study and species or by type of information assessed, and annotated copies that document each finding so that it can be understood, evaluated, and interpreted by the DTT without accessing additional information from study records. Reports will be reviewed by the COR and required changes (if any) made by the Contractor prior to DTT acceptance of the report.

1.6 Reports and Deliverables

Information on specific reporting requirements is presented below. All final reports shall be dated and signed by the individual(s) responsible. Guidelines for submitting reports and deliverables is described in section 2. Electronic report deliverables shall be Section 508 compliant (Sec. 1.6.4). In some cases, there may be accommodations for which the Government will provide information annually.

In addition to the reports mentioned herein, any incident or unforeseeable occurrence not listed here, including physical modifications to the laboratory facilities and personnel changes which may impact study conduct, shall be immediately reported to the Government.

In addition to those documents required by the other terms of this contract, the Contractor shall prepare and submit the following reports to the Contracting Officer’s Representative (COR) electronically (an electronic copy of the Title Page shall also be sent to the Contracting Officer (CO)) as a required deliverable in the manner stated below.

The Contractor shall be responsible for the following reports and deliverables.

Task Status Update to be submitted electronically to the COR every two weeks

Semiannual Reports to be submitted electronically to the CO and COR twice per year

1.6.1 Periodic Reports

1.6.1.1 Sample Work Plans for Each Assessment Type

Upon award, the contractor should provide technical work plans for each of the assessment types described in section 2. The work plans should include the following:

A detailed work plan to explain the proposed approach, procedures, and rationale to reflect a clear understanding of the work described in the Statement and the proposed approach for each aspect, including the conduct and reporting of each assessment type described in the SOW. Information should be provided to demonstrate the understanding of management of time frames for planning and accomplishing the work.

A list of Standard Operating Procedures (SOPs) for conducting each assessment type.

Generation of actual SOPs and subsequent DTT review will be required prior to initiating conduct of assessments under the contract.

A functional organization chart for the project and a list of personnel that will work on the project (including Program Manager, additional professional staff, technical staff and clerical staff).

Include a table listing names of proposed staff, years of experience, proposed role on the project and proposed effort on and availability to the project. The work plan shall be accompanied by the requisite documentation of personnel experience and training.

1.6.1.2 Task Status Report

The Contractor shall provide the COR with a bi‐weekly spreadsheet report that lists assignments and updates to their status. Status shall include the following information: DTT‐quality assessment assignment number, assessment type, test agent, test system, route of exposure, DTTID, laboratory information management system (LIMS) experiment number, date assessment assigned, date started, date original report submitted, date review comments received, date final report submitted, date report accepted.

1.6.1.3 Semiannual Reports

A semiannual report (starting from day one of the contract year) shall be submitted in the format provided by the COR. It shall include a progress report for assignments and activities during the reporting period (using a similar format to the bi‐weekly spreadsheet), a financial statement and graph of proposed vs. actual expenditures and level of effort, personnel changes, and special problem areas. It shall also include a section that summarizes findings across related assignments performed during the reporting period. Suggestions regarding improvement of data and other records generated for DTT studies should be included. One copy of this report shall be submitted to each of the COR and CO.

1.6.2 Research Products

1.6.2.1 Quality Assessment Audit Reports

Quality assessments will be assigned by the DTT Contract Officer’s Representative (COR) based on DTT priorities. The Contractor shall perform each of the assigned assessments (tasks) within the appropriated timeframe. All work shall be handled confidentially and reported for review and acceptance to the COR. The Contractor shall prepare a report for each assessment assigned and performed (see sec. 8 Technical Exhibits for format). Reports shall follow the general format of cover page, table of contents, introduction and background, procedures, assessment findings, list of attachments, conflict of interest statement, and supporting attachments.

Specific formatting requirements will be provided by the COR. Each final assessment report shall be submitted to the COR electronically as a .pdf (Acrobat) through Government furnished program management software (currently Microsoft Dynamic 365: ORBIT). Reports will be reviewed by the COR. If revisions are required by the COR, revised versions of the documents shall be submitted to the Contractor. The COR will notify the project manager when reports (either original or revised) are approved. Assessment reports shall be submitted within the timeframe indicated in the table below.

1.6.3 IT Security Plan and Other IT Deliverables

Refer to Section C, Article TBD of the contract for specific requirements related to IT Security.

1.6.4 508 Compliance The Contractor shall comply with Section 508 of the Rehabilitation Act of 1973 (29 U.S.C. 794d) which requires Federal agencies to purchase Electronic and Information Technologies (EIT) that meet specific accessibility standards. In order to conform to the Section 508 Electronic and Information Technology Accessibility Standards, the Contractor shall perform all work required under this contract in compliance with the provisions delineated in 36 CFR Part 1194 and outlined below.

All electronic deliverables including reports, data entry, web pages, web applications, audio and video shall be Section 508‐compliant; drafts do not need to be compliant. Detailed data that is meant to be processed by a program and not meant to be read by people does not need to be Section 508 compliant. The following Section 508 accessibility technical standards apply to this

SOW:

E205.1 General Electronic content shall comply with E205.

The Contractor shall submit an annual Section 508 report in accordance with the schedule set forth in the ELECTRONIC AND INFORMATION TECHNOLOGY ACCESSIBILITY Article in SECTION H of the contract. Additional information about testing documents for Section 508 compliance, including guidance and specific checklists, by application, can be found at:

https://www.hhs.gov/web/section‐508/index.html and at:

https://www.section508.gov/create/documents , "Create Accessible Documents."

Remediation of any materials that do not comply with the applicable provisions of 36 CFR Part 1194 as set forth in the SOW, shall be the responsibility of the Contractor.

1.7 Post Award Conference/Periodic Progress Meetings

The Contractor shall attend any post award conference convened by the contracting activity or contract administration office in accordance with the Federal Acquisition Regulation (FAR) Subpart 42.5. The CO, COR, and other Government personnel, as appropriate, may meet periodically with the Contractor to review the Contractor's performance. At these meetings the CO will apprise the Contractor of how the Government views the Contractor's performance and the Contractor shall apprise the Government of problems, if any, being experienced.

Appropriate action shall be taken to resolve outstanding issues. These meetings shall be at no additional cost to the Government.

1.7.1 Progress Meetings

Regular progress meetings are not anticipated but may be held as needed. The Contractor shall attend, participate in, and furnish input to scheduled and unscheduled meetings, conferences, and briefings that relate to the functions and services herein as required by the Government to provide effective communication and impart necessary information. The Program Manager or designated representative shall attend meetings as requested by the Government. Meeting attendees shall at times include Contractor managerial, supervisory, and other personnel knowledgeable of the subject matter. Meetings may start or end outside of regular duty hours.

1.8 Contracting Officer’s Representative

As determined by the CO, a COR will be appointed and identified by letter of designation, a copy of which will be provided to the Contractor by the Contracting Officer. The designation letter states the responsibilities and limitations of the COR.

1.9 Personnel

Each requirement described in this SOW shall be performed by an appropriate mix of professional and technical staff. Because there is potential for real and perceived conflicts of interest, neither the organization nor personnel assigned to this contract, including subcontractors or consultants, shall be involved in DTT contracts associated with activities that involve the generation, analysis and/or reporting of DTT data, as these activities are likely to require assessment under this contract. Personnel assigned to this contract with access to federal IT resources may be required to pass a security investigation.

The Contractor shall ensure that all Contractor personnel are adequately trained, possess the required credentials, and are fully qualified and skilled to perform the activities outlined in this SOW. The Contractor shall be responsible for all costs associated with training and necessary travel to ensure that staff maintain required certifications and proficiency to perform assigned duties. Contract personnel shall have the expertise to contribute to problem formulation and research design as required.

Contractor staff designated as key personnel must be approved by the COR. Contractor staff and may need to be available after normal working hours and on weekends. The Contractor shall employ and/or have access to staff with broad professional and technical expertise in environmental health sciences necessary to conduct evaluations of complex scientific literature and fulfill the activities of the SOW.

1.9.1 Professional Personnel

The Program Manager (PM) shall be an employee of the contractor, shall be considered key personnel and member of the professional staff. The project manager shall manage all aspects of the contract (including planning, coordination of effort, scope of the task assignments, and cost analysis), serve as the primary contact for communications with the DTT COR, and serve as an active participant in performance of assessments.

1.9.2 Technical Personnel

The quality assessment support staff shall have education, training, and/or experience necessary to conduct retrospective assessments that evaluate whether summary data and reports from toxicology studies and literature evaluations accurately reflect the study designs, records, and documentation.

1.10 Data Rights, Patents, and Copyrights

The Government has unlimited rights to all documents and materials produced under this contract. All documents and materials, to include the source codes of software, produced under this contract shall be Government owned and are the property of the Government with all rights and privileges of ownership and copyright belonging exclusively to the Government.

These documents and materials may not be used or sold by the Contractor without written permission from the CO. All materials supplied to the Government shall be the sole property of the Government and may not be used for any other purpose. This right does not abrogate any other Government rights.

1.10.1 Source Documentation

All original source documentation is the property of NIEHS. Data summaries and other relevant information generated during the performance of this contract shall be submitted to the COR in the form of status reports and agent specific reports as specified in the Deliverables section above. In addition, annual reports shall be submitted for all activities as specified in the Deliverables section below. To the extent possible, the Government will retain unrestricted access and rights to all tissues, data, and technical information generated under the contract.

1.11 Non‐Disclosure Requirements

Performance under this contract may require the Contractor to access data and information proprietary to a Government Agency, another Government Contractor, or of such nature that its dissemination or use other than as specified in this work statement would be averse to the interests of the Government or others. The Contractor and Contractor’s employees (including Subcontractors), shall not divulge, or release data or information developed, or obtained under performance of this contract, except to authorized Government personnel or upon written approval of the CO. The Contractor shall not use, disclose, or reproduce proprietary data, which bears a restrictive legend, other than as specified in this contract. All documentation showing individual names or other personal information shall be controlled and protected under the provisions of the Privacy Act of 1974, Public Law 93‐579, 5 United States Code (U.S.C.) Section 552a.

1.12 Replication, Dissemination, or use of the Results

It is anticipated that the results of the research conducted under the Contract shall be published in the open, peer‐reviewed scientific literature and/or in a NTP or NIEHS technical report series. Specific details concerning authorship and technical issues surrounding the publication of the research shall be discussed and agreed upon by the PI and the DTT COR on a case‐by‐case basis.

Because premature release of information developed under the contract could be harmful to the public, the Contractor and the Government agree that it is important to consider issues related to the timing of the release of research results when submitting information for publication. With regard to public disclosures of the research to be carried out pursuant to this contract, the Contractor agrees to submit all proposed articles to the COR at least thirty (30) days in advance of submission for publication. Within this thirty‐day period the COR shall inform the Contractor if there are serious concerns raised by the timing of said publication.

Upon receipt of a notification that the Government objects to the publication of said information the Contractor agrees to withhold submission of said publication while the author and the COR discuss the issues of concern. The Contractor agrees to give good faith consideration to all such issues and further agrees that it will not knowingly submit for publication any article, abstract or other item that would be harmful to the public welfare. The final decision on whether to submit an article for publication shall rest with the DTT COR after the procedures set forth above are followed. Similar procedures shall apply for scientific presentations and meeting abstracts.

1.13 Protection of Government and Contract Information

The Contractor shall not cite any information (e.g., contract information, pictures, locations, etc.) obtained through this contract on any hard copy or digital marketing tools to include its company website.

1.14 Non‐Disclosure Statements

The Contractor shall provide signed non‐disclosure agreements to the Government no later than five (5) prior to commencement of work under the contract. Disclosure of information by Contractor personnel may result in removal of Contractor personnel from performance under this contract.

1.15 Safety Laws and Regulations

The Contractor shall comply with FAR 52.236‐13, Accident Prevention (November 1991), In addition to other requirements for accident prevention specified in this contract, the Contractor shall comply with all Federal and State requirements governing safety and health protection.

The Contractor shall indicate its plan for compliance considering conditions of the work described by the design, technical proposal, operational activities of the Government or others in the area of the work, identification of the coordinator, and implementation of all other accident prevention provisions of this contract.

The Contractor shall complete and provide safety notifications as required by national, state, and local regulatory bodies, and will provide documented proof to the CO via the COR, no later than 10 days that such notifications were made.

2 Required Capability Areas

2.1 General Requirements

The base contract consists of several types of retrospective assessments. Primary examples include Assessment of Pathology Tables, Assessment of NTP and NIEHS Reports, Assessment of Laboratory Reports, Assessment of Electronic Information, Assessment of Manuscripts, and Assessment of Monographs. Some assessments may be combinations of the above. Examples of recent NTP and NIEHS products can be found on the public websites (NTP:

https://ntp.niehs.nih.gov/publications, NIEHS: https://www.niehs.nih.gov/research/atniehs/dtt/assoc/reports/niehs‐ reports/index.cfm).

Audit Reports shall follow the general format of cover page, table of contents, introduction and background, audit dates, documents and procedures, audit findings, list of attachments, conflict of interest statement, and supporting attachments. Specific formatting requirements will be provided by the COR.

2.2 Assessment of Pathology Tables

Pathology records, specimens and microscopic diagnoses for a study performed by a Contract Toxicology Laboratory are submitted to the NTP Archives. These pathology materials are generally subjected to peer review by a Pathology Support Contractor (QA Pathologist), and then to peer review by a DTT Pathology Working Group (PWG; information on DTT PWGs in included as supplemental information). Based on recommendations from this peer review process, DTT Pathologists authorize updates of the DTT pathology data to incorporate these consensus diagnoses. Records of the decision to make a change and the original diagnoses are preserved.

In the conduct of an assessment of pathology tables, the contractor shall evaluate the accuracy, consistency, and completeness of all (100%) of the DTT‐authorized diagnostic changes made during the DTT pathology peer review process by assessing the post‐peer review individual animal pathology data against the peer review documentation. This assessment is done before these diagnoses are considered final and before being incorporated into an NTP or NIEHS report and made public. In addition, a subset (~5%) of diagnoses for which no update was indicated shall be assessed, by comparing pre‐peer review tables to post‐peer review tables, to ensure the overall data continuity and stability of unchanged diagnoses. If discrepancies are found in this subset, an extended review may be required. The DTT anticipates utilization of this task to assess molecular pathology data for some studies.

2.3 Assessment of NTP and NIEHS Reports

This type of assessment covers NTP and NIEHS reports of various types. An assessment of an NTP or NIEHS Report evaluates the extent to which data, results, and factual statements presented in a report are provided in an accurate, complete, and consistent manner in comparison to original study records and data that are maintained in the NTP Archives and in Chemical Effects in Biological Systems (CEBS) database.

NTP and NIEHS Reports summarize studies conducted by DTT or NTP partner agencies, that evaluate substances for a variety of cancer and noncancer health‐related effects. These reports provide publicly available data that may be used in identifying potential hazards for human health following environmental or occupational exposure to chemical, physical, and biological agents. While many of these reports describe studies using animals, the DTT anticipates an increase in the publication of other data streams, including data generated from in vitro and in silico studies.

The current report series includes Toxicity Reports (toxicity studies with less than chronic exposure), Technical Reports (typically chronic exposure with evaluation of carcinogenicity), Developmental and Reproductive Toxicity Reports (DART; prenatal and multi‐generational in vivo toxicity studies), and Immunotoxicity Reports (studies evaluating immunotoxicity in model systems). NIEHS reports have a broad scope and may encompass any of the above study designs and are used as a publication mechanism when NTP is not corporate author. Most reports integrate data and results across multiple studies (e.g., rat and mouse subchronic studies). And for many reports, the main studies will have been conducted at a CRO, often under Good Laboratory Practice conditions.

The reports described above are typically prepared in a series of drafts comprising multiple reviews, with support from the DTT Scientific and Information Management contractor.

Throughout the process, DTT personnel provide refinements, clarifications, additions, and revisions to these reports. This process begins after the final data for all studies within a report are available. Typically, an NTP or NIEHS Report is subjected to a quality assessment review prior to being subjected to external peer review; however, earlier versions may be assessed on an as needed basis and with a more limited scope.

Prior to initiation of an NTP or NIEHS Report audit, the study laboratory report(s) and pathology data will have been subjected to their own quality assessment review as described in 2.4 and 2.2, respectively. All findings and discrepancies that were identified during previous assessments of laboratory reports and pathology tables (see below for more information), shall be compared to the NTP or NIEHS Report being assessed and corresponding archival study records to confirm resolution.

Specific details about the design, conduct, and reporting of studies conducted by DTT that will comprise many of the assessments of NTP and NIEHS Reports can be found in the DTT Specifications (direct link to PDF:

https://ntp.niehs.nih.gov/sites/default/files/ntp/test_info/toxspecs/dtt_specs_compiled_508.p

df) and the NTP Immunotoxicity Study Pathology Specifications (https://ntp.niehs.nih.gov/ntp/htdocs/levels/finalntpitoxspecs_508.pdf). Specifications are not currently available for some types of data reported from DTT studies and additional information on these types of studies/endpoints will be provided to the quality assessment support Contractor as necessary.

2.4 Assessment of a Laboratory Report

Each non‐clinical laboratory study or related support activity (chemistry, molecular pathology, etc.) that is performed by a contract toxicology laboratory for DTT results in the submission of a study or task report and a corresponding set of study records that are submitted to the NTP Archives. In an assessment of a laboratory report, the Contractor shall compare the laboratory report to original study records to evaluate the accuracy, consistency, and completeness of all factual (qualitative and quantitative) information presented in the lab report.

The procedure for the assessment of a laboratory report will be utilized for a variety of DTT toxicology study types. The primary types of laboratory reports that will require assessment include those reporting data from toxicity studies, ranging from acute to chronic exposure duration, including neurobehavior, immunotoxicity, cardiovascular, and reproductive/developmental evaluations. DTT studies conducted by contract toxicology laboratories are conducted according to a Government Research outline, the DTT Specifications (see above), and often Good Laboratory Practice Regulations established by the US Food and Drug Administration for Non‐clinical Laboratory Studies. As described above, for some evaluations, there are no available DTT Specifications. In these cases, further information will be provided to the Quality Assessment Support Contractor as necessary.

In an assessment of a laboratory report, the Contractor shall utilize the laboratory report, study records, and data in the DTT LIMS or submitted to CEBS by the toxicology laboratory. The Contractor shall begin with a familiarizing inspection and inventory of the prestart chemistry, in‐life, and necropsy phases of the study records (e.g., chemistry, toxicology and animal husbandry), which usually start with the first shipment of test agent to the laboratory and continue through final necropsy of the last animal. If records are incomplete, the Quality Assessment Support Contractor shall contact the DTT COR prior to finalizing the assessment;

the COR will arrange for records to be provided to the Quality Assessment Support Contractor if available. In addition, the contractor shall also assess the laboratory report against relevant contractual standards (for example the appropriate DTT Specifications and DTT‐prepared Research Outline). Specific requirements and procedures for assessing individual study reports may vary with their content.

As described above in Section 2.2, histopathology data typically undergo an assessment following pathology peer review and are not generally within scope of the laboratory report assessment.

The toxicology and carcinogenesis studies and ancillary activities that shall be assessed involve predominantly the disciplines of chemistry, animal care, toxicology, molecular toxicology, and clinical and anatomic pathology. The following list is not intended to be all‐inclusive, but rather indicates the variety of interdisciplinary procedures, records, data, and results that quality assessment support contractor staff shall evaluate during assessments of NTP and NIEHS Reports.

(a) Test‐article (TA) identity, purity, analytical method performance, homogeneity, and stability

(b) Dose formulation/preparation, analysis, and stability under storage and exposure conditions

(c) Exposure generation/monitoring (inhalation studies)

(d) Animal receipt, quarantine, randomization, and general husbandry including control of environmental conditions in animal rooms or chambers and feed storage area; sentinel animal information

(e) In life toxicology parameters such as dam/litter endpoints during gestation/lactation, body weight, food/drinking water consumption, clinical signs of toxicity, developmental landmarks, clinical pathology (clinical chemistry, hematology, urinalysis, etc.), neurotoxicity/neurobehavioral endpoints, functional reproductive (fertility) and immunotoxicity endpoints (disease resistance, hypersensitivity)

(f) Biological sample collection/analysis (collection/analysis of blood, tissues and excreta for determination of characterization/fate/reactivity of the test agent; tissue distribution/burden, metabolism, toxicokinetics)

(g) Biochemical and molecular endpoints, including molecular analysis of tumor and non‐tumor tissue

(h) Genetic toxicity data

(i) Necropsy, gross evaluations/pathology, organ weights, histology and histopathology, cell proliferation

(j) Teratology data

(k) Cardiotoxicity and physiology data

(l) Neurotoxicity and neurobehavior data

(m) Adequacy of procedures used to capture and process data, data trails

(n) Statistical analysis of group data for data not acquired (e.g., cardiotoxicity and neurotoxicity) using a DTT‐sponsored LIMS.

(o) in vitro evaluations

For most of the data and information included in the report, a 100% assessment shall be conducted. For certain large data sets, sampling strategies shall be applied. These include a 10‐ 20% data assessment (examples include individual animal dose‐volume for gavage and dermal studies, inhalation exposure data, etc.) or an assessment of control and high dose data with a scan of other groups for obvious discrepancies (e.g., toxicity endpoint data that are not collected via Provantis; current examples include physiology data, biological sample analysis data, biochemical assay data, etc.). Before a sampling strategy is applied, all the pertinent data shall be scanned to identify possible omissions or other obvious discrepancies that may require adaptation of the sampling scheme or indicate that an increase, up to 100% assessment, is necessary. Similarly, findings encountered during the assessment of sampled data shall trigger a more extensive assessment of the data if deemed necessary by the Contractor in consultation with the DTT COR. In either case, the nature and incidence of discrepancies will dictate whether the assessment needs to be expanded. On a routine basis, data collected via Provantis (currently survival, body weight, clinical pathology observations, organ weights, reproductive and development data, and histopathology) and summarized in an NTP or NIEHS Report based on original, individual animal data collected via Provantis shall be assessed using summary reports. However, individual animal reports shall be used as necessary to verify information in a report (e.g., the list of tissues collected for histopathology would be assessed using individual animal pathology data). Therefore, both summary and individual animal Provantis data shall be used during the conduct of assessments and the DTT may on occasion request that the Contractor compare summary data to individual animal data.

2.5 Assessment of Electronic Information

The DTT makes a wide range of information available electronically. Most electronic information (EI) is available as HTML Files, PDF documents, XML files or downloadable files through the NTP public website (http://ntp.niehs.nih.gov) or related websites (examples include but are not limited to http://cebs.niehs.nih.gov and http://tools.niehs.nih.gov/ntp_tox/index.cfm).

In an assessment of electronic information, the contractor shall assess the accuracy, consistency, and completeness of electronic information by comparing information from a primary source to other known publicly available instances of the same information. A common use of this assessment is in the review of inputs and outputs to support the development of new data management and sharing pipelines by DTT. DTT also requires support for validation and acceptance for LIMS systems used on GLP studies. DTT anticipates that the use and types of electronic information will be expanded during the period of performance and may include conduct of various assessments related to DTT databases and websites.

A variety of ancillary support studies or tasks (i.e., molecular pathology, in vitro assays, etc.) are conducted in connection with DTT studies, either on samples resulting from the main study or as separate supporting experiments. Each of these results in the submission of a corresponding report and set of records. Some examples include but are not limited to the following:

analytical chemistry conducted at a laboratory separate from the in‐life laboratory such as for characterization of test agent identity and purity; analysis of tissue for test agent or metabolites; toxicokinetic studies; molecular pathological assessments; in vitro analyses;

evaluation of laboratory information management system data; other supplemental studies or analysis of results. Assessments of these activities shall be conducted similar to laboratory report assessments, adapting the assessment process as needed based on the specific types of study information.

2.6 Assessment of Manuscripts

The DTT produce manuscripts describing studies conducted both internally and at CROs. For those conducted internally, study records, data, and statistical outputs are housed in CEBS. For studies conducted at CROs, study records are stored at the NTP Archives while the data is housed in CEBS. DTT also publishes information from literature evaluation studies in peer review manuscripts and those assessments are described below in Section 2.7.

In an assessment of a manuscript for a study conducted internally, the contractor shall evaluate the accuracy, consistency, and completeness of both the data and the text of the manuscript.

Evaluation of data shall be done by comparing the data conveyed in figures and tables against those provided in CEBS.

In an assessment of a manuscript for a study conducted at a CRO, the contractor shall compare information with the original study records and assess the figures tables and text for accuracy, consistency, and completeness. The process for these assessments will be similar in size and scope to those described above in Section 2.2.

2.7 Assessment of Monographs and Other Literature Review Products In addition to the conduct of in vitro and in vivo studies, the DTT performs systematic literature evaluations and develops and applies innovative informatic approaches to produce fit‐for‐ purpose literature assessments. Examples of literature‐based evaluations include scoping reviews, systematic evidence maps, and systematic reviews.

Cancer and Noncancer Health Effect evaluations are described in more detail on the NTP website:

Cancer: https://ntp.niehs.nih.gov/go/aboutroc Noncancer: https://ntp.niehs.nih.gov/go/noncancer

Literature evaluations may be published in a variety of formats, including Government reports

(https://ntp.niehs.nih.gov/go/monographs), manuscripts and others.

For these publications, the contractor shall review the accuracy, consistency, and completeness of the factual information presented in the specified publication or product by comparing it to data included in the original/cited literature and reviewing for internal consistency.

3 Operating Procedures

3.1 Project Management

The Program Manager shall be responsible for the daily management of the contract and serve as points of contact for communication with the Government officials. The Program Manager is expected to be knowledgeable and involved in all aspects of the contract, including project status, record keeping, reporting, and detailed analysis of budget for all contract functions.

Over the course of a given project, the Government anticipates that the processes used to develop documents and research products and carry out related activities may change as program‐related changes are implemented. Additionally, updates to the instructions for a specific activity may be made by the COR due to changing Government requirements or for particular cases due to the unique or complex nature of a particular assignment. Any updates to processes or instructions would be communicated by the COR to the Contractor.

The Contractor shall provide program management support for the contract to include but not limited to:

Ensuring that the timeframes for each milestone are met Submitting deliverables in a timely manner Providing accurate invoices on time Managing personnel and staffing Providing project management and tracking Maintaining the transparency in operations Providing timely communication with the COR

The Contractor shall use a DTT Program and project management (PPM) software (i.e., Microsoft Dynamics 365) specified by the Government to document and update contract activities, including, but not limited to, research outlines submitted by the Government, technical work plans and budget information, milestone schedule, work status, interim data (in the form of tables, graphs, spreadsheets, and/or other documents), and deliverables. All substantive communication and decisions throughout the performance of work on an activity shall be documented within the software.

Any incident or unforeseeable occurrence not listed in this document, or any change in personnel that might have an impact on fulfilling the requirements as described in the SOW shall be reported immediately to the CORs.

3.2 Data Management

The Contractor shall provide a plan for the management, sharing, backup, and security of all paper and electronic data and information collected and generated as part of completing the products outlined in this SOW. This plan shall be approved by the COR. The Contractor shall keep all data and documents used or prepared confidential until Government publication.

3.3 Subcontractors & Consultants

If subcontractors/consultants are proposed, the offeror must clearly define how any proposed subcontract/consultant arrangements would meet program needs. A letter of commitment from any proposed subcontractors/consultants must be provided. If subcontracting is suggested, details must be given as to how this would be structured, organized, managed, and monitored.

3.4 Contractor Travel

Memorandum M‐19‐21 directs all Federal agencies to transition recordkeeping to a fully electronic environment and ensure all Federal records be created, retained, and managed in electronic formats. Therefore, contractors will be provided electronic access to databases and archives where this information is stored. However, until the transition of all paper records to electronic records is complete, there will be occasional instances when study files exist as paper records or CDs/DVDs and audits will have to be conducted onsite. In these cases, the assignments will necessitate travel between the Contractor’s office or remote assignment and NTP Archives in Research Triangle Park, NC. As a result, the proximity of the quality assessment support contractor’s main or remote offices to the NTP Archives will significantly facilitate travel to the NTP Archives as approved by the COR.

Quality assessment support contractors may occasionally be asked to accompany the COR to meetings with DTT and/or contractor personnel involved in generation, analysis and/or reporting of data for the NTP. These meetings would be at locations such as the DTT offices, NTP Archives, or other locations as directed by the COR.

3.5 Contract Management and Monitoring

Contract performance will be monitored through periodic site visits, meetings with the Program Manager, review of semiannual reports, task status and audit reports, electronic mail, and telephone discussion, as required. DTT site visits may be conducted to inspect facilities, project data, and records. The COR will monitor the quality of the audit reports.

4 Government Furnished Property and Services

4.1 Supplies and Services

The Government shall provide the Contractor access to the following.

- NIEHS and NIH electronic resources including, but not limited to, Citrix and Virtual

Private Network (VPN) client software to access the NIH and NIEHS networks.

- Access to study records including laboratory reports, Pathology Working Group records, NTP Reports, study data and records, and selected web applications.

- Access to equipment needed to make electronic copies of study records and data (when required and at the NTP Archives or DTT offices).

- Access to the NTP Archives and DTT offices.

- Electronic access to and training on a DTT LIMS.

4.2 Facilities

Assessments of NTP Reports and laboratory reports may occasionally require access to study records that are stored at the NTP Archives. Other activities, including assessments of pathology tables and electronic information, report preparation for all assignments, and filing, will generally be performed at the quality assessment support contractor’s facilities.

4.3 Software/Training

The Government will provide licensed access to a Government‐specified program management software (currently Microsoft Dynamic 365: ORBIT). DTT will provide contract staff with the necessary training in person or via videocast prior to accessing the system.

4.4 Equipment

The contractor shall provide personal computer facilities necessary to communicate electronically with the COR or others associated with DTT testing, to access/generate laboratory information management reports needed to conduct most tasks, to generate Section 508 compliant deliverables and to conduct assessments of electronic information. The Government will not provide computing equipment or IT support to the Contractor for use at its facility.

5 Contractor Furnished Property and Services

5.1 General

The Contractor shall provide space and facilities necessary to perform the work described in this SOW. The Contractor shall identify sources and furnish information necessary to fulfill this contract, such as source data and all necessary specifications for databases covered for the project as well as the necessary information pertaining to the website maintenance.

5.2 Equipment and Software

The Contractor is required to supply all equipment, property, and services needed to perform the work described in this SOW that are not explicitly specified as being provided by the Government. These materials include but are not limited to computers, printers, phones, Microsoft Office, Web‐based conferencing software, and Adobe Acrobat Professional (for Section 508 compliance). Equipment such as computers and printers shall support Internet and Intranet connections, be compatible with NIH computer operating systems, and comply with NIH security policies. The Contractor shall conform to security requirements or usability guidelines as appropriate.

The Contractor shall maintain compatible versions of software applications and licensure and shall provide personnel properly trained with the latest releases and upgrades. Further, the Contractor shall provide a plan for maintaining computer skills as upgrades and computer program enhancements become available.

6 Definitions and Acronyms

6.1 Definitions

Although not inclusive of every term used within this SOW, the following provides a list of definitions used throughout this SOW and commonly used in the acquisition field.

Contracting Officer (CO) – a person with authority to enter into, administer, and or terminate contracts, and make related determinations and findings on behalf of the Government. Note:

The only individual who can legally bind the Government.

Contracting Officer's Representative (COR) – As defined in DFARS 202.101, means an individual designated and authorized in writing by the Contracting Officer to perform specific technical or administrative functions. Any individual delegated responsibilities pursuant to subpart FAR 1.602‐2 of Reference (e), regardless of local terminology, must be certified in accordance with this instruction. For example, local terminology can be COR, Contracting Officer’s Technical Representative, Technical Point of Contact, Technical Representative, Alternate COR, Administrative COR, Assistant COR, Line‐Item Manager, Project Manager, Quality Assurance Personnel, Quality Assurance Evaluator, or COR management. These other surveillance support personnel may serve as on‐site representatives of the COR in performance of actual contract surveillance if they meet all COR requirements and have been appointed by the Contracting Officer as alternate CORs.

Contractor – a supplier or vendor awarded a contract to provide specific supplies or service to the Government. The term used in this contract refers to the prime.

Contractor‐Acquired Property – property acquired, fabricated, or otherwise provided by the Contractor for performing a contract, and to which the Government has title.

Day – unless otherwise specified, a calendar day.

Deliverable – anything that can be physically delivered but may include non‐manufactured things such as meeting minutes or reports.

Government‐Furnished Property – property furnished to the Contractor by the Government and subsequently furnished to the Contractor for performance of a contract. Government‐ furnished property includes, but is not limited to, spares and property furnished for repair, maintenance, overhaul, or modification.