Attachment 3_SOW_Base Contract_Final.pdf

STATEMENT OF WORK

February 28, 2024

NIAID EVALUATION SERVICES

1. BACKGROUND AND INTRODUCTION

The National Institute of Allergy and Infectious Diseases (NIAID) conducts and supports basic and applied research to better understand, treat, and ultimately prevent infectious, immunologic, and allergic diseases. For over 70 years, NIAID research has led to new therapies, vaccines, diagnostic tests, and other technologies that have improved the health of millions of people in the United States and around the world.

NIAID has a unique mandate, which requires the Institute to respond to emerging public health threats. Toward this end, NIAID manages a complex and diverse research portfolio that aims to do the following:

• Expand the breadth and depth of knowledge in all areas of infectious, immunologic, and allergic diseases

• Develop flexible domestic and international research capabilities to appropriately respond to emerging and re- emerging disease threats at home and abroad

NIAID advances the understanding, diagnosis, and treatment of many of the world's most intractable and widespread diseases. Key research areas include newly emerging and re-emerging infectious diseases such as COVID-19, tuberculosis and influenza, HIV/AIDS, biodefense, and immune-mediated diseases including asthma and allergy.

This Contract will support the evaluation functions of the Policy, Planning, and Evaluation (PP&E) Branch and NIAID. PP&E provides institutional leadership for systematic, trans-NIAID strategic planning, evaluation, and policy analysis to support and advance NIAID's mission. NIAID conducts a myriad of evaluations, including feasibility and evaluation design studies, needs assessments, process evaluations, and outcome and impact evaluations. This acquisition is designed to support the program evaluation and management functions of PP&E which includes the following.

• Oversee the development and implementation of an ongoing strategic planning process

• Analyze the NIAID research portfolio to inform policy decisions and science management

• Direct ongoing and special assessments of research programs to guide the development of future Institute programs and initiatives

• Develop briefing materials for NIAID staff use in preparation for meetings with and presentations to the NIH Director, federal agencies, Congress, and scientific organizations and advocacy groups

• Prepare timely and accurate responses to Congress, HHS, other federal agencies, and NIH stakeholders who request information about the NIAID research portfolio

• Coordinate the NIAID institutional response for draft federal rule making documents, guidance, and policy documents

• Monitor, analyze, and develop position statements on key policies and regulations that affect the NIAID mission and/ or its constituents

2. SCOPE

The purpose of this contract is to provide design and implement program evaluation projects for NIAID’s scientific research and management programs as well as provide support for evaluation training and evaluation related activities.

Evaluation projects at NIAID often include feasibility studies, needs assessments, process evaluations, outcome and impact evaluations, and related evaluation support and training for NIAID staff. The descriptions of the Tasks that follow offer examples of the types of support that have been required in the past and should be anticipated for this contract. However, not all work performed by NIAID can be anticipated, and the work that might be required under this contract is not limited to these examples.

A. Task Ordering

1. This contract provides for the issuance of Task Orders on a negotiated basis. Task orders may be Fixed Price or Time and Materials.

2. Task Orders will be issued as needed based on NIAID's evaluation priorities and may be comprised of any combination of the types of services described within this Statement of Work. The Contractor will be required to respond to Requests for Task Order Proposals (RFTOPs) within five business days of issuance by the Government Contracting office.

3. For each Task Order, the Contractor will schedule and attend an initial planning meeting with the COR, the designated Task Leader, and appropriate NIAID and contract staff as necessary.

Work objectives and expectations will be addressed during this meeting as well as identification of any potential problems and proposed solutions. The initial meeting will take place within 10 business days of initiation of each task order.

3. GENERAL DESCRIPTION OF TECHNICAL REQUIREMENTS

Independently, and not as an agent of the Government, the contractor shall furnish all the necessary services, qualified personnel, material, equipment, and facilities not otherwise provided by the Government as needed to fulfill the Statement of Work as follows using their best efforts.

A. Evaluation Design

Contractor provides staff trained in program evaluation of biomedical and behavioral science research programs and management programs. These staff shall work with the NIAID to develop a tailored, detailed, and rigorous design for program evaluation projects. As applicable for each individual project, this will include working in a team setting with the NIAID staff to:

a) Assess the needs of evaluation stakeholders and incorporate these needs into the evaluation design;

b) Identify specific short-term and long-term outcome measures to guide program evaluation efforts;

c) Identify topics or program features important to incorporate in a specific program evaluation;

d) Review previous related evaluation reports and other background information to inform the evaluation design;

e) Develop specific, detailed, measurable and objective program evaluation questions;

f) Identify data sources suitable to answering the evaluation questions;

g) Select the appropriate analytical methods, and;

h) Develop a detailed evaluation plan to implement the evaluation according to the priorities and needs of the NIAID, the availability of data, and the resources required for the evaluation.

B. Preparation of Background and Summary Papers

Contractor prepares summary reports and/or background papers drawn from available materials describing the scientific research or management program to be evaluated. Sources for such background papers may include web sites, NIH progress reports and grant applications, NIH Requests for Applications or Program Announcements, scientific research publications, internal NIAID initiative proposals, conference summaries, and other sources.

The NIAID and the Contractor will work together to develop a tailored outline for each background paper required, and the Contractor will independently synthesize and summarize this information to fully inform the design and implementation of the evaluation research project. The background papers shall be written and edited so as to be well-organized, accurate, at the appropriate level of detail, in one voice, and in plain English. The Contractor may be asked to update the paper at regular intervals throughout the project to facilitate final reporting.

C. Qualitative Interviews & Analyses

Contractor provides personnel with the qualifications and experience needed to plan and carry out qualitative inquiry, including qualitative interviews.

The Contractor Shall:

a) Develop specific, detailed protocols for structured, semi-structured, and unstructured interviews as required for the evaluation project;

b) Ensure that each protocol addresses (but does not merely repeat) the evaluation questions; minimizes social desirability, recall, and other biases;

and is easily and fully understood and accessible to all potential respondents.

c) Assist with the preparation of all documentation for Paperwork Reduction Act (PRA)/Office of Management and Budget (OMB) and Institutional Review Board (IRB) clearance, as required;

d) Conduct the interview process, keeping the evaluation team apprised of progress on a weekly or bi-weekly basis. Interviews may require travel, as determined by the COR.

e) Use discretion, tact, and diplomacy to maximize the interviewees' confidence in the NIAID and the evaluation process;

f) Prepare written summaries of interviews, or arrange for transcription, as agreed with the evaluation project team;

g) Summarize the results of interviews and analyze according to the analysis plan designed by the team; suggest and employ specific qualitative analysis tools in consultation with NIAID staff;

h) Generate a written report synthesizing the interview process, outcomes, and recommendations.

Contractor understands that interview data remain the exclusive property of the Government.

D. Focus Groups & Analyses

The Contractor shall provide personnel with the qualifications and experience needed to plan and carry out focus groups. Focus groups may relate to scientific or administrative programs, for example. Additional focus group topics may be identified later by NIAID.

a) Develop specific, detailed protocols for the conduct of focus groups, including scripts and invitations, as required for the evaluation project;

b) Provide a trained focus group facilitator;

c) Ensure that each protocol addresses (but does not merely repeat) the evaluation questions; minimizes social desirability, recall, and other biases, and is easily and fully understood and accessible to all potential respondents;

d) Assist with the preparation of all documentation for Paperwork Reduction Act (PRA)/Office of Management and Budget (OMB) and Institutional Review Board (IRB) clearance, as required;

e) Conduct the focus group process, keeping the evaluation team apprised of progress on a weekly or bi-weekly basis. Focus Groups may require travel, as determined by the COR.

f) Take into account especially sensitive issues or situations and use discretion, tact, and diplomacy to maximize participant confidence in the NIAID and the evaluation process; prepare written summaries of focus group comments, or arrange for transcription, as agreed with the evaluation project team;

g) Summarize the results of focus groups according to the analysis plan designed by the team;

h) Generate a written report synthesizing the focus group process, outcomes, and recommendations.

E. Surveys & Analyses

The Contractor shall provide personnel with the qualifications and experience needed to plan and carry out surveys in a variety of modalities, including telephone, email, and internet surveys. Surveys may relate to scientific or administrative programs, for example. Additional survey topics may be identified later by NIAID. Personnel must have expertise in developing both open-ended and close-ended survey questionnaires.

The Contractor shall:

a) Develop specific, detailed survey questions as required for the evaluation project;

b) Ensure that all surveys are appropriately pre-tested;

c) Ensure that each questionnaire addresses (but does not merely repeat) the evaluation questions; minimizes social desirability, recall, and other biases; and is easily and fully understood and accessible to all potential respondents;

d) Assist with the preparation of all documentation for Paperwork Reduction Act (PRA)/Office of Management and Budget (OMB) and Institutional Review Board (IRB) clearance, as required by the evaluation;

e) Implement the survey process, keeping the evaluation team apprised of progress on a weekly or bi-weekly basis, as determined by the evaluation team.

f) Use discretion, tact, and diplomacy to maximize the interviewees' confidence in the NIAID and the evaluation process

g) Provide the NIAID with a copy of the survey responses in spreadsheet or database acceptable to the entire evaluation team.

h) Generate a written report synthesizing the survey process, outcomes, and recommendations.

i) Prepare summaries of survey results and analyses as agreed with the evaluation project team according to the analysis plan

Contractor understands that survey data remain the exclusive property of the Government.

F. Data Abstraction

The Contractor shall provide personnel with the qualifications and experience needed to plan and carry out the abstraction of qualitative and quantitative data from a variety of sources, for example web sites, NIH grant applications and progress reports, financial and/or budget reports, resumes and CVs, NIH Requests for Applications or Program Announcements, scientific research publications, internal NIAID initiative proposals, and conference summaries.

The evaluation team, including the Contractor staff, will develop specific, detailed protocols and data abstraction procedures as required for the evaluation project.

This will include detailed and rigorous quality control procedures to maximize intra-and inter-rater reliability.

a) Implement the data abstraction process, keeping the evaluation team apprised of progress on a weekly or bi- weekly basis;

b) Implement the quality control procedures developed by the team, alerting the COR to any problems encountered as soon as possible;

c) Provide the NIAID with a copy of the abstraction results in spreadsheet or database acceptable to the entire evaluation team;

d) Prepare summaries of data abstraction results as agreed with the evaluation project team according to the analysis plan.

Abstraction data will remain the exclusive property of the Government.

G. Scientific Literature Review, Bibliometrics, and Outcome

Analysis

As required, the contractor will conduct searches, categorize and prioritize publications, and prepare reviews of the scientific research literature on topics related to individual program evaluation projects. These reviews will require critical analysis of the scientific literature, with special attention to research methods, populations or animal models studied, and outcome measures used.

Using criteria developed by the NIAID evaluation team, the Contractor will select publications to be included and perform descriptive bibliometric and other outcome analyses, such as patents, product development and treatment guidelines using measures agreed upon by the evaluation team.

Scientific literature reviews, bibliometric analyses, and outcome analyses may relate to scientific or administrative programs, for example. Additional topics may be identified later by NIAID.

a) Prepare summaries of bibliometrics and outcome analyses results as agreed with the evaluation project team, according to the analysis plan;

b) Generate a written report synthesizing the information and data from the review and analyses.

H. Economic Analysis

The Contractor will provide personnel with the qualifications and experience needed to plan and carry out analyses of the economic, health, and societal impact of biomedical research programs. Examples include: the impact of NIH and NIAID supported research on wages and/or employment of the scientific research workforce, and the overall labor force; the economic impact of NIH and NIAID research programs on universities and other recipients of funds; the macro- and microeconomic impact of the small business set-aside programs administered by the NIH and the NIAID; the effect of NIH and NIAID training and career development programs on wages and employment of the scientific workforce; the effect of input pricing on the proposed costs of research applications submitted to the NIH and the

NIAID; the effect of alternative programmatic and staffing models on the cost of conducting clinical research; the economic impact of public health advances, such as improvements in health and quality of life, and springing from specific NIH and NIAID research.

a) Ensure that economic analyses are conducted with rigorous study designs, and that bias in the evaluation questions and methods is minimized to the greatest extent.

b) Generate a written report synthesizing the information and data from the analyses.

I. Evaluation Training

The Contractor shall provide staff with experience in developing and providing training to scientific and non-scientific staff on program evaluation methodology, practices, and interpretation and visualization of results. These staff shall work with the NIAID to develop a tailored and detailed plan for evaluation-related trainings.

In an effort to enhance general knowledge of evaluation principles and practices, the Contractor will develop trainings that will provide a means to educate NIAID personnel on evaluation topics and practices (e.g., best practices, evaluation theory, evaluation methodologies).

For each training session, the Contractor shall develop different training objectives, based on the priorities of NIAID. Trainings may vary in their format, but the Contractor shall develop each training session in a format that is relevant to the topic, relevant to NIAID staff, provides specific examples of applying the course content to their positions at NIAID, and is interactive. The trainings may be instructor-led online (e.g., via MS Teams, Zoom) or through self-paced electronic training modules. Alternatively, the Contractor may conduct trainings in a NIAID conference room, in-person. Audience size may range from ten to 100 personnel from across all functions of NIAID and/or NIH; and training may vary in length (e.g., 2 hours, 4 hours, 8 hours, 2 days).

In order to conduct a training session, the Contractor must be a subject matter expert in the course content. The Contractor will meet with NIAID and submit a training plan within 15 calendar days of Task award. Two weeks after approval of the training plan, the Contractor shall submit a course outline and presentation slides to NIAID for review and final approval. The Contractor shall conduct the training for NIAID staff within 60 calendar days of NIAID approval of training plans.

J. Analysis of NIH Data

Contractor shall plan and carry out the downloading and analysis of NIH data from various databases and systems, for example Query, View, Report (QVR) and RePORTER.

The evaluation team, including the Contractor staff, will

a) Develop specific, detailed procedures for the retrieval and analysis of NIH data as required for the evaluation project. This will include detailed and rigorous quality control procedures;

b) Provide the NIAID with a copy of the data in spreadsheet or database acceptable to the entire evaluation team.

c) Prepare summaries of data results as agreed with the evaluation project team according to the analysis plan;

d) Generate a written report synthesizing the information and data from the analyses.

All data will remain the exclusive property of the Government.

K. Writing and Report Preparation

The Contractor shall prepare written reports and PowerPoint slides summarizing the results of evaluation projects. Reports include annual reports, monthly status reports, executive summaries, and other reports and graphs, as requested by NIAID.

Reports may require different levels of writing, from highly technical to plain language; and will require synthesizing and interpreting complex scientific information. Each report should be clear, accurate, free of errors, and visually appealing.

The Contractor shall provide reports in electronic format, but hard copy may also be required. The Contractor will be responsible for ensuring that reports are prepared in electronic formats that are fully compatible with accessibility requirements, including but not limited to the requirements of Section 508 of the Rehabilitation Act.

L. Other Evaluation Support Services

NIAID may require additional support with other evaluation related activities. For example, the Contractor may be asked to develop components of an evaluation such as evaluation tools (e.g., a survey or logic model), design and implement a database, conduct a literature review, conduct bibliometric analyses, develop and design training on evaluation topics that may not lead to an entire evaluation.

4. MANAGEMENT AND ADMINISTRATION

4.1 Reports and Deliverables

All reports required herein shall be submitted in electronic format. Hard copies may also be requested.

All electronic reports submitted shall be compliant with Section 508 of the Rehabilitation Act of 1973. Additional information about testing documents for Section 508 compliance, including guidance and specific checklists, by application, can be found at: http://www.hhs.gov/web/508/index.html under "Making Files Accessible."

All paper/hardcopy documents/reports submitted under this contract shall be printed or copied, double- sided, on at least 30 percent post-consumer fiber paper, whenever practicable.

http://www.hhs.gov/web/508/index.html

4.1.1 Progress Reports.

In addition to the required reports set forth elsewhere within the Statement of Work, the preparation and submission of regularly recurring Technical Progress Reports will be required. These reports will require descriptive information about the activities undertaken for each Task Order during the reporting period and will require information about planned activities for future reporting periods.

The following Technical Progress Reports will be required:

a) Monthly Status Reports.

The Contractor will submit a narrative progress report and a financial report for the previous month. The narrative report will contain a description, by task, of the Contractor’s program activities; Current status of each ongoing Project Assignment; Proposed activities for the upcoming period; anticipated deadlines; any problems during the past month, or anticipated during the upcoming month, and proposed solutions. This report will be delivered to the COR. A copy of the narrative for each Project Assignment will also be sent to the Task Leader for that task, if requested. In addition, the Contractor may be required to give timely and accurate reports on the status of any Project Assignment at any point in a given month at the request of the COR or CO.

b) Annual Reports.

On a yearly basis, beginning thirty calendar days after the one-year anniversary of the Task Order award, the Contractor will submit to the CO and COR an Annual Report that includes a description of the significant activities and accomplishments that have occurred in the preceding year, problems encountered, and the resolution of those problems; a statistical summary of all tasks (cumulative costs, cost analyses, and trends) from the past year.

c) Final Report and Presentation Slides.

At the conclusion of the Contract, the Contractor will submit a Final Report, including an Executive Summary and Presentation Slides that summarize the highlights and accomplishments for the duration of the Contract. The Report will not exceed 30 typed pages (not including attachments), unless otherwise requested by the COR, and describe the evaluation methodology, data analysis, and a discussion of the key findings and recommendations. The Report will contain an overview as well as a section providing details for each of the Task Areas and all subtask areas. Statistical summaries of all activities (cumulative costs, cost analyses and trends) will be reported by year. The Final Report will also detail any problems encountered and the resolution of those problems, as well as recommendations and conclusions based on experience and the results obtained over the course of the contract.

4.1.2 Other Reports/Deliverables

The Contractor shall provide all deliverables in Microsoft Office 2021 or higher format (e.g., Microsoft 365), or in a format agreed upon, in advance, by the COR. All written deliverables will be phrased in layperson language. Statistical and other technical terminology will not be used without providing a glossary of terms. Additionally, all deliverables required under this contract shall be submitted online via a secure file transfer (e.g., OneDrive, eRDS) in accordance with the Government specifications.

At a minimum, all deliverables shall be marked with the contract number and contractor name. The Contractor shall guarantee that all required materials shall be delivered in immediate usable and acceptable condition.

a) Program Management Plan.

Within ten business days of the Contract award, the Contractor will submit a Program Management Plan (PMP) to the COR that will describe and provide guidance to Contract staff involved with the management and administration of the contract. The PMP will ensure the Contract staff complies with all terms and conditions that govern the Contract. The PMP shall include an executive summary of the roles and responsibilities of the contracting staff, identify who is responsible for various contract administration activities, and be flexible and adapt to changing circumstances.

b) Financial Status Report.

The Contractor shall submit a Financial Status Report at the Task Order level to the COR. The spending plan must be revised and resubmitted any time there is a change to the prices, spending plan, or scope of work (due to a contract amendment).

c) Work Plan.

For each new Project Assignment, the Contractor will schedule and attend an initial meeting with the COR, the designated Task Leader, and appropriate NIAID and contract staff. This meeting will be for the purpose of discussing the Project Assignment objectives, the expectations and requirements of the NIAID, potential problems and cost estimates. The Contractor will finalize the Plan and forward it to the COR and CO for approval within five business days.

d) Meeting Minutes.

The Contractor shall deliver a meeting summary capturing action items, next steps, and overview of meeting highlights.

e) List of all Open Projects.

The Contractor shall provide a spreadsheet, listing all open projects and a brief status update for each project at the Project Manager/COR bi-weekly meeting.

f) Data Collection Instruments.

As required by the project and requested by the COR, the Contractor shall develop data collection instruments within a timeframe specified by the COR. The Contractor shall make revisions to the data collection instruments after receipt of Government comments.

g) PRA/OMB Clearance and IRB Packages.

As required by the project and requested by the COR, the Contractor shall complete and revise, in collaboration with the NIAID Paperwork Reduction Act (PRA) liaison, clearance packages for the PRA/Office of Management and Budget (OMB) and Institutional Review Board (IRB) within a timeframe specified by the

COR.

h) Reports and Graphs.

The Contractor shall provide reports and graphs requested by the COR. The Contractor shall make revisions to the packages after receipt of Government comments.

i) Completion Report & Transition Plan.

Upon completion of each Task Order, the Contractor will prepare and submit to the CO (with a copy to the COR) documentation that the Project Assignment is completed and the total costs. The report will list Project Assignment title, Project Manager, start and completion dates, brief description of the Project

Assignment, a detailed account of all services provided, records of any problems experienced, and a detailed breakdown of costs associated with the performance of the Project Assignment. One copy of all documents written, edited, or designed by the Contractor under the Project Assignment will be attached. All data files, data, documentation, disks, tapes, recordings, programming code, and other material produced, as part of the Project Assignment will be delivered to the COR or her/his designee. At the end of the contract, the Contractor shall deliver a Transition Plan indicating how ongoing activities will be transferred, lessons learned, personnel and staffing needs as well as skill and expertise;

transportation of all deliverables listed under the deliverable and reporting requirements section of the contract; unused materials and supplies; contractor developed database programs, entries, and related files; deployment schedule, and all other files requested by the NIAID COR.

4.3. TRANSITION

The Contractor shall participate in transition activities associated with the orderly transfer of service responsibilities both at contract start-up and immediately prior to contract expiration.

(END OF STATEMENT OF WORK for the BASE CONTRACT)

4.3. TRANSITION