NIAID Eval SSN Final.pdf

11/17/2023 Page 1 of 11

Sources Sought

Sources Sought Reference No: HHS-NIH-NIAID- 75N93024R00005

Agency/Office

Department of Health and Human Services, National Institutes of Health, NIAID

Contracting Office Address

Office of Acquisitions

Division of Extramural Activities, NIAID

5601 Fishers Lane, Room 3D10, MSC 9821

Bethesda, MD 20892-9821

Type Support Services Sources Sought

Solicitation Number

HHS-NIH-NIAID- 75N93024R00005

Title NIAID Evaluation Services

Classification Code R707 – Support- Management: Contract/Procurement/Acquisition Support

NAICS Code 541611 – Administrative Management and General Management Consulting Services

Posted Date: November 17, 2023 Response Date: December 4, 2023

Primary Point of Contact Matt Lear, Contract Specialist Matt.lear@nih.gov 240-669-5109

Secondary Point of Contact Rosemary Gomes, Contracting Officer/Team Lead rosemary.gomes@nih.gov 240-669-5374 mailto:Matt.lear@nih.gov mailto:rosemary.gomes@nih.gov

11/17/2023 Page 2 of 11

Introduction This is a non-Research and Development (Non-R&D) Sources Sought notice. This is NOT a solicitation for proposals, proposal abstracts, or quotations. The purpose of this notice is to obtain information regarding the availability and capability of all qualified sources, including small and small disadvantaged businesses, to perform a potential non R&D service requirement.

Background The National Institute of Allergy and Infectious Diseases (NIAID) conducts and supports basic and applied research to better understand, treat, and ultimately prevent infectious, immunologic, and allergic diseases. For almost 70 years, NIAID research has led to new therapies, vaccines, diagnostic tests, and other technologies that have improved the health of millions of people in the United States and around the world. NIAID has a unique mandate, which requires the Institute to respond to emerging public health threats. Toward this end, NIAID manages a complex and diverse research portfolio that aims to do the following:

• Expand the breadth and depth of knowledge in all areas of infectious, immunologic, and allergic diseases

• Develop flexible domestic and international research capabilities to appropriately respond to emerging and re- emerging disease threats at home and abroad

NIAID advances the understanding, diagnosis, and treatment of many of the world's most intractable and widespread diseases. Key research areas include newly emerging and re-emerging infectious diseases such as COVID-19, tuberculosis and influenza, HIV/AIDS, biodefense, and immune-mediated diseases including asthma and allergy.

This Contract will support the evaluation functions of the Policy, Planning, and Evaluation (PP&E) Branch and NIAID. PP&E provides institutional leadership for systematic, trans-NIAID strategic planning, evaluation, and policy analysis to support and advance NIAID's mission. NIAID conducts a myriad of evaluations, including feasibility and evaluation design studies, needs assessments, process evaluations, and outcome and impact evaluations. This acquisition is designed to support the program evaluation and management functions of PP&E which includes the following:

• Oversee the development and implementation of an ongoing strategic planning process

• Analyze the NIAID research portfolio to inform policy decisions and science management

• Direct ongoing and special assessments of research programs to guide the development of future Institute programs and initiatives

• Develop briefing materials for NIAID staff use in preparation for meetings with and presentations to the NIH Director, federal agencies, Congress, and scientific organizations and advocacy groups

11/17/2023 Page 3 of 11

• Prepare timely and accurate responses to Congress, HHS, other federal agencies, and NIH stakeholders who request information about the NIAID research portfolio

• Coordinate the NIAID institutional response for draft federal rule making documents, guidance, and policy documents

• Monitor, analyze, and develop position statements on key policies and regulations that affect the NIAID mission and/ or its constituents

This project is a renewal of the ongoing contract (75N93019D00035), with Ripple Effect Communications, that is set to expire on July 29, 2024.

Purpose and Objectives

The purpose of this NIAID Evaluation Services Sources Sought Notice is to discern whether or not there are other contractors, including small or small disadvantaged businesses, capable of and interested in performing the work described herein. The NIAID does not intend to award a contract on the basis of responses received nor otherwise pay for the preparation of any information submitted.

As a result of this Sources Sought Notice, the NIAID may issue a Request for Proposals (RFP) if it deemed necessary and appropriate. THERE IS NO SOLICITATION AVAILABLE AT THIS TIME. However, should such a requirement materialize, no basis for claims against NIAID shall arise as a result of a response to this Sources Sought Notice or the NIAID’s use of such information as either part of our evaluation process or in developing specifications of any subsequent requirement.

Project Requirements

The purpose of this contract is to provide design and implement program evaluation projects for NIAID’s scientific research and management programs as well as provide support for evaluation training and evaluation related activities.

A. Evaluation Design

Contractor provides staff trained in program evaluation of biomedical and behavioral science research programs and management programs. These staff shall work with the NIAID to develop a tailored, detailed, and rigorous design for program evaluation projects. As applicable for each individual project, this will include working in a team setting with the NIAID staff to:

a) Assess the needs of evaluation stakeholders and incorporate these needs into the evaluation design;

b) Identify specific short-term and long-term outcome measures to guide program evaluation efforts;

c) Identify topics or program features important to incorporate in a specific program evaluation;

d) Review previous related evaluation reports and other background information to inform the evaluation design;

11/17/2023 Page 4 of 11

e) Develop specific, detailed, measurable and objective program evaluation questions;

f) Identify data sources suitable to answering the evaluation questions;

g) Select the appropriate analytical methods, and;

h) Develop a detailed evaluation plan to implement the evaluation according to the priorities and needs of the NIAID, the availability of data, and the resources required for the evaluation.

B. Preparation of Background and Summary Papers

Contractor prepares summary reports and/or background papers drawn from available materials describing the scientific research or management program to be evaluated. Sources for such background papers may include web sites, NIH progress reports and grant applications, NIH Requests for Applications or Program Announcements, scientific research publications, internal NIAID initiative proposals, conference summaries, and other sources.

The NIAID and the Contractor will work together to develop a tailored outline for each background paper required, and the Contractor will independently synthesize and summarize this information to fully inform the design and implementation of the evaluation research project. The background papers shall be written and edited so as to be well-organized, accurate, at the appropriate level of detail, in one voice, and in plain English. The Contractor may be asked to update the paper at regular intervals throughout the project to facilitate final reporting.

C. Qualitative Interviews & Analyses

Contractor provides personnel with the qualifications and experience needed to plan and carry out qualitative inquiry, including qualitative interviews.

The Contractor Shall:

a) Take the lead, working with the NIAID in a team setting, for developing specific, detailed protocols for structured, semi-structured, and unstructured interviews as required for the evaluation project;

b) Ensure that each protocol addresses (but does not merely repeat) the evaluation questions; minimizes social desirability, recall, and other biases; and is easily and fully understood and accessible to all potential respondents.

c) Assist with the preparation of all documentation for Paperwork Reduction Act (PRA)/Office of Management and Budget (OMB) and Institutional Review Board (IRB) clearance, as required;

d) Conduct the interview process, keeping the evaluation team apprised of progress on a weekly or bi-weekly basis. Interviews may require travel, as determined by the COR.

e) Take into account especially sensitive issues or situations and use discretion, tact, and diplomacy to maximize the interviewees' confidence in the NIAID and the evaluation process;

f) Prepare written summaries of interviews, or arrange for transcription, as agreed with the evaluation project team;

11/17/2023 Page 5 of 11

g) Summarize the results of interviews and analyze according to the analysis plan designed by the team; suggest and employ specific qualitative analysis tools in consultation with NIAID staff;

h) Generate a written report synthesizing the interview process, outcomes, and recommendations.

Contractor understands that interview data remain the exclusive property of the Government.

D. Focus Groups & Analyses The Contractor shall provide personnel with the qualifications and experience needed to plan and carry out focus groups. Focus groups may relate to scientific or administrative programs, for example.

Additional focus group topics may be identified later by NIAID.

a) The Contractor will take the lead, working with the NIAID in a team setting, for developing specific, detailed protocols for the conduct of focus groups, including scripts and invitations, as required for the evaluation project;

b) Provide a trained focus group facilitator;

c) Ensure that each protocol addresses (but does not merely repeat) the evaluation questions; minimizes social desirability, recall, and other biases, and is easily and fully understood and accessible to all potential respondents;

d) Assist with the preparation of all documentation for Paperwork Reduction Act (PRA)/Office of Management and Budget (OMB) and Institutional Review Board (IRB) clearance, as required;

e) Conduct the focus group process, keeping the evaluation team apprised of progress on a weekly or bi-weekly basis. Focus Groups may require travel, as determined by the COR.

f) Take into account especially sensitive issues or situations and use discretion, tact, and diplomacy to maximize participant confidence in the NIAID and the evaluation process; prepare written summaries of focus group comments, or arrange for transcription, as agreed with the evaluation project team;

g) Summarize the results of focus groups according to the analysis plan designed by the team;

h) Generate a written report synthesizing the focus group process, outcomes, and recommendations.

Contractor understands that focus group data remain the exclusive property of the Government.

E. Surveys & Analyses

The Contractor shall provide personnel with the qualifications and experience needed to plan and carry out surveys in a variety of modalities, including telephone, email, and internet surveys. Surveys may relate to scientific or administrative programs, for example. Additional survey topics may be identified later by NIAID. Personnel must have expertise in developing both open-ended and close-ended survey questionnaires.

The Contractor shall:

11/17/2023 Page 6 of 11

a) In collaboration with NIAID, assume lead responsibility for developing specific, detailed survey questions as required for the evaluation project;

b) Ensure that all surveys are appropriately pre-tested;

c) Ensure that each questionnaire addresses (but does not merely repeat) the evaluation questions; minimizes social desirability, recall, and other biases; and is easily and fully understood and accessible to all potential respondents;

d) Assist with the preparation of all documentation for Paperwork Reduction Act (PRA)/Office of Management and Budget (OMB) and Institutional Review Board (IRB) clearance, as required by the evaluation;

e) Implement the survey process, keeping the evaluation team apprised of progress on a weekly or bi-weekly basis, as determined by the evaluation team.

f) Take into account especially sensitive issues or situation and use discretion, tact, and diplomacy to maximize the interviewees' confidence in the NIAID and the evaluation process

g) Provide the NIAID with a copy of the survey responses in machine- readable form-spreadsheet or database-acceptable to the entire evaluation team.

h) Generate a written report synthesizing the survey process, outcomes, and recommendations.

i) Prepare summaries of survey results and analyses as agreed with the evaluation project team according to the analysis plan

Contractor understands that survey data remain the exclusive property of the Government.

F. Data Abstraction

The Contractor shall provide personnel with the qualifications and experience needed to plan and carry out the abstraction of qualitative and quantitative data from a variety of sources, for example web sites, NIH grant applications and progress reports, financial and/or budget reports, resumes and CVs, NIH Requests for Applications or Program Announcements, scientific research publications, internal NIAID initiative proposals, and conference summaries.

The evaluation team, including the Contractor staff, will develop specific, detailed protocols and data abstraction procedures as required for the evaluation project. This will include detailed and rigorous quality control procedures to maximize intra- and inter-rater reliability.

a) Implement the data abstraction process, keeping the evaluation team apprised of progress on a weekly or bi- weekly basis;

b) Implement the quality control procedures developed by the team, alerting the COR to any problems encountered as soon as possible;

c) Provide the NIAID with a copy of the abstraction results in machine- readable form-spreadsheet or database-acceptable to the entire evaluation team;

d) Prepare summaries of data abstraction results as agreed with the evaluation project team according to the analysis plan.

Abstraction data will remain the exclusive property of the Government.

11/17/2023 Page 7 of 11

G. Scientific Literature Review, Bibliometrics, and Outcome Analysis

As required, the contractor will conduct searches, categorize and prioritize publications, and prepare reviews of the scientific research literature on topics related to individual program evaluation projects.

These reviews will require critical analysis of the scientific literature, with special attention to research methods, populations or animal models studied, and outcome measures used.

Using criteria developed by the NIAID evaluation team, the Contractor will select publications to be included and perform descriptive bibliometric and other outcome analyses, such as patents, product development and treatment guidelines using measures agreed upon by the evaluation team.

Scientific literature reviews, bibliometric analyses, and outcome analyses may relate to scientific or administrative programs, for example. Additional topics may be identified later by NIAID.

a) Prepare summaries of bibliometrics and outcome analyses results as agreed with the evaluation project team, according to the analysis plan;

b) Generate a written report synthesizing the information and data from the review and analyses.

H. Economic Analysis

The Contractor will provide personnel with the qualifications and experience needed to plan and carry out analyses of the economic, health, and societal impact of biomedical research programs. Examples include: the impact of NIH and NIAID supported research on wages and/or employment of the scientific research workforce, and the overall labor force; the economic impact of NIH and NIAID research programs on universities and other recipients of funds; the macro- and microeconomic impact of the small business set-aside programs administered by the NIH and the NIAID; the effect of NIH and NIAID training and career development programs on wages and employment of the scientific workforce; the effect of input pricing on the proposed costs of research applications submitted to the NIH and the NIAID; the effect of alternative programmatic and staffing models on the cost of conducting clinical research; the economic impact of public health advances, such as improvements in health and quality of life, and springing from specific NIH and NIAID research.

a) Ensure that economic analyses are conducted with rigorous study designs, and that bias in the evaluation questions and methods is minimized to the greatest extent.

b) Generate a written report synthesizing the information and data from the analyses.

I. Evaluation Training

The Contractor will provide staff with experience in developing and providing training to scientific and non-scientific staff on program evaluation methodology, practices, and interpretation and visualization

11/17/2023 Page 8 of 11 of results. These staff shall work with the NIAID to develop a tailored and detailed plan for evaluation-related trainings.

In an effort to enhance general knowledge of evaluation principles and practices, the Contractor will develop trainings that will provide a means to educate NIAID personnel on evaluation topics and practices (e.g., best practices, evaluation theory, evaluation methodologies).

For each training session, the Contractor will develop different training objectives, based on the priorities of NIAID. Trainings may vary in their format, but the Contractor shall develop each training session in a format that is relevant to the topic, relevant to NIAID staff, provides specific examples of applying the course content to their positions at NIAID, and is interactive. The trainings may be instructor-led online (e.g., via MS Teams, Zoom) or through self-paced electronic training modules.

Alternatively, the Contractor may conduct trainings in a NIAID conference room, in-person. Audience size may range from ten to 100 personnel from across all functions of NIAID and/or NIH; and training may vary in length (e.g., 2 hours, 4 hours, 8 hours, 2 days).

In order to conduct a training session, the Contractor must be a subject matter expert in the course content. The Contractor will meet with NIAID and submit a training plan within 15 calendar days of Task award. Two weeks after approval of the training plan, the Contractor shall submit a course outline and presentation slides to NIAID for review and final approval. The Contractor shall conduct the training for NIAID staff within 60 calendar days of NIAID approval of training plans.

J. Analysis of NIH Data

Contractor shall plan and carry out the downloading and analysis of NIH data from various databases and systems, for example Query, View, Report (QVR) and RePORTER.

The evaluation team, including the Contractor staff, will

a) Develop specific, detailed procedures for the retrieval and analysis of NIH data as required for the evaluation project. This will include detailed and rigorous quality control procedures;

b) Provide the NIAID with a copy of the data in machine-readable form-spreadsheet or database-acceptable to the entire evaluation team.

c) Prepare summaries of data results as agreed with the evaluation project team according to the analysis plan;

d) Generate a written report synthesizing the information and data from the analyses.

All data will remain the exclusive property of the Government.

K. Writing and Report Preparation

The Contractor shall prepare written reports and PowerPoint slides summarizing the results of evaluation projects. Reports include annual reports, monthly status reports, executive summaries, and other reports and graphs, as requested by NIAID.

Reports may require different levels of writing, from highly technical to plain language; and will require synthesizing and interpreting complex scientific information. Each report should be clear, accurate, free of errors, and visually appealing.

11/17/2023 Page 9 of 11

The Contractor shall provide reports in electronic format, but hard copy may also be required. The Contractor will be responsible for ensuring that reports are prepared in electronic formats that are fully compatible with accessibility requirements, including but not limited to the requirements of Section 508 of the Rehabilitation Act.

L. Other Evaluation Support Services

NIAID may require additional support with other evaluation related activities. For example, the Contractor may be asked to develop components of an evaluation such as evaluation tools (e.g., a survey or logic model), design and implement a database, conduct a literature review, conduct bibliometric analyses, develop and design training on evaluation topics that may not lead to an entire evaluation.

Anticipated Period of Performance

It is anticipated that this will be an Indefinite Delivery/Indefinite Quantity (IDIQ) with fixed-price Task Orders with an estimated award date on or about July 30, 2024.

Capability statement / information sought

Capability statements submitted as a result of this announcement should demonstrate the offeror’s qualification, expertise and experience, specifically providing evidence as to the capability to immediately perform this requirement. Capability statements should clearly convey information regarding the respondent’s capabilities, including: (1) staff expertise, including their availability, experience and formal and other training; (b) current in-house capability and capacity to perform the work; (c) prior completed projects of a similar nature; (d) corporate experience and management capability to deliver comprehensive services of this scope and magnitude; and (e) examples of prior completed Government contracts, references, and other related information.

Interested Contractors must submit a capability statement (five page limitation, excluding resumes) describing their company’s experience and ability to immediately perform this effort, which includes the following: (1) a summary list of similar work previously performed; (2) the professional qualifications and specific experience of staff who may be assigned to the requirement; (3) resumes for proposed key personnel, including the Principal Investigator/co-PI, which reflect education, and previous work relevant to the proposed requirement; (4) a general description of the facilities and other resources needed to perform the work; and (5) demonstrated ability to carry out the work. Capability Statements must not include links to internet web site addresses (URLs) or otherwise direct readers to alternate sources of information. Font size must be 10 to 12 points. Spacing must be no more than 15 characters per inch. Within a vertical inch, there must be no more than six lines of text. Print margins must be at least one-inch on each edge of the paper.

Each response should include the following Business Information:

a. UEI number

b. Company Name

c. Company Address

11/17/2023 Page 10 of 11

d. Company Points of Contact (both technical and administrative), including names, titles, addresses, telephone and fax numbers and Email addresses

e. Current GSA Schedule appropriate to this Sources Sought

f. Do you have a Government-approved accounting system? If so, please identify the agency that approved the system.

g. Type of Company (i.e., large business, non-profit, educational institute, small business, 8(a), woman owned, veteran owned, etc.), as validated via the System for Award Management (SAM). All offerors must be registered in the SAM, located at http://www.sam.gov/

Responses must be submitted no later than 3:00 PM (E.S.T.), December 4, 2023. Capability statements will not be returned and will not be accepted after the due date. These statements should be submitted electronically (via e-mail) to Matt Lear, Contract Specialist, at matt.lear@nih.gov and Rosemary Gomes, Contracting Officer/Team Lead, at rosemary.gomes@nih.gov. These statements should be submitted in Adobe Portable Document Format (PDF); however, capability statements in Microsoft Word will also be accepted. The e-mail subject line must specify HHS-NIH-NIAID-75N93024R00005. Facsimile responses will not be accepted.

Common Cut-off Date:

Electronically submitted, tailored capability statements are due not later than 3:00 PM (E.S.T.), on December 4, 2023. CAPABILITY STATEMENTS WILL NOT BE RETURNED AND WILL NOT BE ACCEPTED

AFTER THE DUE DATE.

Disclaimer and Important Notes

This notice does not obligate the Government to award a contract or otherwise pay for the information provided in response. The Government reserves the right to use information provided by respondents for any purpose deemed necessary and legally appropriate. Any organization responding to this notice should ensure that its response is complete and sufficiently detailed to allow the Government to determine the organization’s qualifications to perform the work. Respondents are advised that the Government is under no obligation to acknowledge receipt of the information received or provide feedback to respondents with respect to any information submitted. After a review of the responses received, a pre-solicitation synopsis and solicitation may be published in Federal Business Opportunities.

However, responses to this notice will not be considered adequate responses to a solicitation.

The Government will not entertain questions regarding this Market Research; however, general questions may be emailed to the following addresses:

Contract Specialist: Matt Lear

Email address: matt.lear@nih.gov

Contracting Officer/Team Lead: Rosemary Gomes

Email Address: Rosemary.Gomes@nih.gov http://www.sam.gov/ mailto:matt.lear@nih.gov mailto:rosemary.gomes@nih.gov mailto:matt.lear@nih.gov mailto:Rosemary.Gomes@nih.gov

11/17/2023 Page 11 of 11

Confidentiality

No proprietary, classified, confidential, or sensitive information should be included in your response.

The Government reserves the right to use any non-proprietary technical information in any resultant solicitation.