HRSA E-E IDIQ - SECTION L and M Final.pdf

HRSA Evidence-Building and Evaluation Services IDIQ; Solicitation #75R60224R00003

SECTION L - INSTRUCTIONS, CONDITIONS AND NOTICES TO OFFERORS

L.1 General Instructions

1. Your attention is directed to the requirements for the technical, past performance, business proposal information, and Voluntary Product Accessibility Template (VPAT) to be submitted in accordance with the following instructions establishing the acceptable minimum requirements for the format and content of proposals. Proposals must be complete at the time of proposal submission. Failure to submit a complete proposal at the time proposals are due may result in the exclusion of your proposal from further consideration by the Government for award of a contract.

Offerors are advised that the Government intends to make this award without discussions.

If necessary, the Contracting Officer may conduct oral and/or written discussions with all firms that have a reasonable prospect of award and, therefore, are included in the competitive range. Because the Government intends to make award without discussions, the initial offer should contain the Offeror’s best terms and prices.

Offeror’s must have a valid SAM registration when submitting proposals in order to be evaluated for award. Offeror’s with expired or nonexistent SAM registrations will be deemed unresponsible in accordance with FAR Part 9.104-1 and FAR Part 4.1102 and will NOT have their proposals evaluated.

2. Proposals shall be submitted single spaced, printable on 8.5 x 11 inch paper, paginated front to back, 12 point font (Times New Roman), 1-inch margins, with searchable features in the technical proposal and business proposal. 10 point font (Times New Roman) will be allowed for tables and graphics. 8, 9 or 10 point font (Times New Roman) will be allowed for headers and footers.

Each page shall be numbered, and each volume shall be tabbed to correspond to the table of contents. The Offeror may include foldouts up to 11 x 17 inches only to display graphics, flow charts, organizational charts, or drawings, which will not be counted towards the page limit for the technical volume.

The Business Proposal shall be submitted with Excel compatible formatted content with formulas preserved as an appendix.

3. Where data and/or information appears in one part, it does not have to be repeated in any other part; however, it shall be cross referenced by indicating the specific location including the volume and page number as a minimum. The clarity, relevance, and conciseness of the proposal is important, not the length.

4. This RFP does not commit the Government to pay any cost for preparing and submitting a proposal. In addition, the Contracting Officer is the only individual who can legally commit the Government to the expenditure of public funds in connection with this proposed acquisition.

5. HHS’s or HRSA’s letterhead and/or logo should not be included in the Offeror’s proposal. The Offeror’s proposals also should not include proposals from past contracts.

6. If proprietary information is being proposed, please mark as required under the FAR.

7. A statement indicating whether or not any exceptions are taken to the terms and conditions of the request for proposal (RFP) shall be part of the proposal transmittal letter. Any exceptions taken must include identification of the specific paragraphs and rationale for each exception. Exceptions shall also be noted in the specific volume of the proposal, at the location of the exception.

8. The proposal shall be prepared in accordance with FAR 52.215-1, Instruction to Offerors.

Proposals shall be submit as stated below to be considered responsive to this RFP (no exceptions). Proposals shall include an initial page with the information required by the clause at paragraph (c)(2) and shall be submitted in four (4) volumes as outlined below:

Volume Title

I IDIQ Technical Proposal II Past Performance III Price Proposal

IV VPAT

The Technical, Past Performance, Price, VPAT Proposal volumes/information shall be submitted separately and clearly labeled. Each of these parts shall be separate and complete in itself so that the evaluation of one may be accomplished independently of the evaluation of the other.

9. Offerors will submit a single Volume I IDIQ Technical Proposal.

10. Offerors shall adhere to the following page limits, with appendices and attachments excepted:

a. Volume I IDIQ Technical Proposal – 70 Pages

b. Volume II Past Performance – 25 Pages

c. Volume III Price Proposal –

d. Volume IV Sub- VPAT – 25 Pages

11. The Offeror’s proposal must stipulate that it is predicated upon all the terms and conditions of this RFP. In addition, it must contain a statement to the effect that it is firm for a period of at least 180 days from the date of receipt by the Government.

12. The Offeror must submit proposals and data comprehensive enough to provide the basis for a sound evaluation.

13. Proposals must be signed by an official authorized to bind your organization and must be submitted by 2:00 PM EDT, January 8, 2024 by email in PDF and Excel-compatible format to HRSAIDIQ@hrsa.gov. Offerors are encouraged to request a delivery receipt to serve as evidence of timely receipt of its proposal by the government.

L.2 Inquiries

1. All questions and / or inquiries concerning this solicitation must be submitted via email to HRSAIDIQ@hrsa.gov by Noon EDT, December 11, 2023. Questions submitted after the due date will not be accepted. Any resulting addition, deletion, or change to the RFP document will be made by issuing a formal amendment, which may be downloaded from http://www.SAM.gov. Offerors are instructed specifically to contact only the issuing contract office in connection with any aspect of this requirement prior to contract award.

All questions concerning this solicitation must be submitted in the following format:

# Question Section of RFP Page # Government Response (To Be

Provided)

Questions submitted in a different format will not receive a Government Response.

2. The government will post all questions received from Offerors and federal staff in response to this solicitation and associated responses on the SAM.GOV website at www.sam.gov via formal amendment. Any addition, deletion, or change to the RFP document will also be made by issuing a formal amendment, which may be downloaded from www.sam.gov. It is the Offerors responsibility to monitor SAM.GOV or documents related to this solicitation.

L.3 Volume I IDIQ Technical Proposal

1. Volume I shall not contain references to price/cost; however, resource information such as data concerning labor hours and categories, materials, subcontracts, etc., must be contained in the technical proposal so that your understanding of the Statement of Work may be evaluated. It must disclose your technical approach in sufficient detail to provide a clear and concise presentation that includes the requirements of the technical criteria.

2. Offerors shall submit a single complete Volume I IDIQ Technical Proposal. Volume I

IDIQ Technical Proposal shall not exceed 70 pages, excluding attachments, addendums, resumes (individual resumes shall not exceed two (2) pages) and other required appendices.

mailto:HRSAIDIQ@hrsa.gov mailto:HRSAIDIQ@hrsa.gov http://www.sam.gov/

A. Technical Approach to Hypothetical Projects

The Offeror shall provide a proposed technical approach to the four hypothetical projects listed below. This approach should include a detailed project plan that reflects an understanding of, and ability to address the key technical skills and capacities required to execute the proposed projects, including a description of Offeror’s approach to each of the identified project components.

If needed, the Offeror is authorized to generate hypothetical answers to any assumptions and/or outstanding questions relevant to these hypothetical projects that may be required to develop the response.

The Offeror shall describe the project management, staffing and planning approach to adequately staff, organize and manage the tasks required to address the hypothetical evaluation requirements. The project management, staffing and planning approach shall indicate how organizational role and responsibilities will be divided, decisions made, work monitored, and quality and timeliness of products/services ensured.

If the Offeror proposes to use any consultants or subcontractor employees to carry out elements within the requirements, the management and staffing plan shall specify how the Offeror, consultants, and/or subcontractors will work together, how the tasks and activities will be coordinated and how quality assurance and timeliness of products/services will be accomplished. The Offeror shall identify all facilities and equipment available for the completion of all contract requirements.

The Offeror shall include up to three (3) relevant examples of recent (within past 5 years) evaluation studies (as defined in Appendix A of the statement of work), other evidence-building activities (as defined in Appendix C of the Statement of Work), communication and dissemination activities and products, and data optimization project experiences, in which the Offeror addressed similar question(s)/situation(s) to the hypothetical project, relative to public health programs similar to those administered by HRSA.

3. In order to establish uniformity and to facilitate the evaluation, Offerors shall submit a

Volume I IDIQ Technical Proposal organized in the following sections:

Factor 1 – Technical subfactor 1 – Understanding of the Problem subfactor 2 – Technical Approach subfactor 3 – Management and Staffing Plan subfactor 4 – Personnel subfactor 5 – Organizational Experience

Factor 2 – Hypothetical Proposals

4. Factor 1 -Technical

Subfactor 1 – Understanding the Problem

The Offeror shall demonstrate a knowledge and understanding of the SOW requirements and the work that may be included in future task order. The Offeror’s technical proposal shall, in the Offeror’s own words, provide its understanding of the following:

a. The objectives, organization, and operation of HRSA as a public health agency of the federal government.

b. HRSA programs and the Offeror’s capability in addressing the unique issues inherent to each program.

c. Major program and policy areas and related issues around health care and public health topics that or will have implications for HRSA program areas over the next five years.

Subfactor 2 – Technical Approach

The Offeror’s technical approach shall address the following areas:

a. How the Offeror will meet each of the requirements specified in the SOW.

b. The Offeror’s understanding of and ability to address the various technical assistance and consultative services covered in the SOW.

c. The Offeror’s technical capacity to address key technical skills and capacities that may be required in future RFTOP, including providing specific examples.

d. How future task orders may be organized, staffed, and managed.

Subfactor 3 – Management and Staffing Plan

The Offeror’s management and staffing plan shall address the following areas:

a. Submit a detailed project plan indicating how each aspect of the SOW and future task orders will be accomplished.

Project management activities include the following:

1) Management of personnel.

2) Policies and procedures for effective operations and contract management.

3) Coordination and management of subcontractors as appropriate.

4) Establishment of effective communications and reporting procedures with the

Task Order COR.

5) Proper financial management of the contract/task orders.

6) Other project management tasks necessary to successfully meet or exceed the requirements of this contract/task orders.

7) Risk management and document control.

b. Describe how the Offeror will adequately staff, organized, and manage future task orders, including how organizational roles and responsibilities will be divided, how decisions will be made, how work will be monitored, and how the Offeror will ensure the quality and timeliness of products and services.

c. Describe how the Offeror’s will enable it to start projects within fourteen (14) days of task order award, move efficiently and conduct multiple tasks concurrently, complete complex tasks within narrow timeframes, and assure quality of services.

d. Describe how the Offeror will coordinate obtaining expert consultants for future task orders.

e. Describe how the Offeror plans on developing, monitoring, reviewing, and adding new expertise to the consultant pool, as well as the method of maintaining a pool of qualified experts that may meet the need for a potential of more than 1,000 three-person site visits per year.

f. Describe how the Offeror will provide training to consultants and/or subcontractors prior to commencing work under task orders.

g. Describe how the Offeror will ensure staff are trained in Federal regulations, policies, program expectations, measures to enhance effectiveness of consultation, approaches to dealing with diversity and conflict, and other topics relevant to program administration.

h. Describe how the Offeror will eliminate a consultant for noncompliance or poor performance.

i. The management and staffing plan shall also address:

1) Management structure and approach.

2) Initial staffing plan and how the Offeror will meet future staffing requirements.

3) Consultant and subcontractor management.

4) Data security.

5) Plans and procedures for the administration of task orders.

6) Procedures for communicating with the CO and task order COR regarding project status and details on reporting and record keeping, as detailed in the SOW.

7) Procedures for communicating and handling problems and potential problems including recommending potential solutions.

j. If consultants or subcontractors shall be used to carry out elements within the solicitation and future task orders, the plan shall specify how the Offeror and the consultants/subcontractors shall coordinate and work together to accomplish tasks and how quality assurance shall be accomplished.

k. The staffing plan shall define employee and consultant recruitment and long-term retention methods, including a break-down of when an employee or consultant is:

full-time permanent employee, part-time employee, temporary employee, term-of-contract personnel, independent consultant, and subcontractor.

l. Describe how the Offeror will recruit staff for specialized skills that are required under task orders and the methodology for obtaining a pool of experts that possess experience and expertise as listed in Section J, Attachment C and have expert knowledge of HRSA programs.

m. Identify facilities and equipment, including evidence of the availability of adequate facilities, space, and equipment necessary to accomplish project goals, objectives, and future task orders.

n. A statement of understanding on the required use of the government-furnished TA Tracking System (TATS).

Subfactor 4 – Personnel

The Offeror’s personnel proposal shall address the following areas:

a. Provide the names, educational preparation, years of experience, types of professional experience, and technical accomplishments of personnel proposed.

b. Proposed staff, identified by the labor categories specified in Section J Attachment B.

c. Offerors shall note if they will not fill specific labor categories.

d. Curriculum Vitae or resumes and commitment letters for all key proposed personnel, including a sample of fifty (50) staff or expert consultants who can address the various type of technical and consultative assistance that may be required in future task orders.

Subfactor 5 – Organizational Experience

The Offeror’s Organizational Experience proposal shall include the following information:

• Program Title.

• Name of Contracting Agency/Commercial Activity.

• Contract Number/Identification Number.

• Award Date.

• Contract Type (FFP, FPIF, CPIF, etc.).

• Awarded Amount.

• Name, mailing address, telephone number, fax number and email address of the customer COR for each Government contract, or the contract administrator for each major subcontract/commercial contract.

• Contract Period of Performance.

• Relevancy: Describe the relevancy of the past or present contract/subcontract to this solicitation. Relevancy discussion may also include end item comparison with respect to use or complexity, size and scope of the effort.

• Provide any other information that the Offeror deems important to understand the organizational experience on the contract/subcontract, the Offeror’s ability to deliver quality products and services, meet schedule/performance requirements, and achieve customer satisfaction.

The Offeror’s organizational experience proposal shall address the following areas:

a. The Offeror’s experience staffing and starting a task order within as little as two weeks of award.

b. Specific experience with and knowledge of HRSA programs.

c. A statement of the mission/purpose, general background, experience, and qualification of the Offeror.

d. Information on similar or related contracts, subcontracts, or grants which the Offeror has completed.

e. Experience that demonstrates technical ability and subject matter expertise such that the Offeror can provide a wide variety of technical assistance.

f. Experience managing resources and personnel effectively.

g. Detailed description of the Offeror’s system for ensuring that all documents and presentations that would be prepared under this contract and future task orders would be comprehensively reviewed and edited for quality, accuracy, and thoroughness.

Factor 2 Hypothetical Technical Proposals

3. HRSA reserves the right to award any of the hypotheticals provided as the first task order under this IDIQ. All technical proposals submitted in response to the hypotheticals should be submitted as if the proposal will result in an award.

Hypothetical Projects

The Offeror shall provide a proposed technical approach to the four hypothetical projects listed below. This approach should include a detailed project plan that reflects an understanding of, and ability to address the key technical skills and capacities required to execute the proposed projects, including a description of Offeror’s approach to each of the identified project components.

If needed, the Offeror is authorized to generate hypothetical answers to any assumptions and/or outstanding questions relevant to these hypothetical projects that may be required to develop the response.

The Offeror shall describe the project management, staffing and planning approach to adequately staff, organize and manage the tasks required to address the hypothetical evaluation requirements. The project management, staffing, and planning approach shall indicate how organizational role and responsibilities will be divided, decisions made, work monitored, and quality and timeliness of products/services ensured.

If the Offeror proposes to use any consultants or subcontractor employees to carry out elements within the requirements, the management and staffing plan shall specify how the Offeror, consultants, and/or subcontractors will work together, how the tasks and activities will be coordinated and how quality assurance and timeliness of products/services will be accomplished. The Offeror shall identify all facilities and equipment available for the completion of all contract requirements.

The Offeror shall include up to three (3) relevant examples of recent (within past 5 years) evaluation studies (as defined in Appendix A of the statement of work), other evidence-building activities (as defined in Appendix C of the Statement of Work), communication and dissemination activities and products, and data optimization project experiences, in which the Offeror addressed similar question(s)/situation(s) to the hypothetical project, relative to public health programs similar to those administered by HRSA.

HRSA IDIQ Hypotheticals

Hypothetical Project #1: Impact Evaluation Project

The Offeror shall prepare an impact evaluation of the Health Centers program.

Community health centers, located throughout the United States, provide primary health care services to underserved populations, racially and ethnically diverse populations, individuals with income below 200% of the Federal Poverty Guideline, individuals with no insurance or on Medicaid).

The program's goals are to provide access to quality healthcare services, increase the utilization of services by the target population, to maximize its financial reimbursement opportunities, and improve health outcomes among the population it serves.

The impact evaluation developed in response to this hypothetical program should assume a two-year timeframe and address each of the following project requirements:

• Identify a representative sample of health centers on which to conduct the evaluation;

use appropriate sampling methodology to reduce selection bias.

• Develop an evaluation framework including an evaluation plan and evaluation questions to address the impact of the health centers program on its recipients/patients.

• Identify and develop an approach to evaluate program operations, and specifically consider the question, "Is the program designed to achieve its stated goals in the most efficient and effective manner possible?"

• Describe an approach for identifying and engaging relevant stakeholders, including people with lived experience, on the evaluation design, key questions, and distribution of findings.

• Specify program outcome(s) to be evaluated, as well as an approach to determine the extent to which changes in those outcomes can be attributed to the health center program.

• Assess the impact of the program’s goals and objectives over time and examine the factors associated with improved patient behaviors, health center utilization, and health center patient outcomes.

• Identify performance and quality measure(s) to define how success is being measured for achieving program outcomes as well as for use by HRSA for required annual reporting through the Government Performance and Results Modernization Act of 2010.

• Assess the health centers’ capacity to collect and report data and implement strategies related to the evaluation of program operations.

• Develop a data collection and data analysis plan that describes primary data collection and incorporates secondary data sources.

• Propose data collection efforts that align with parallel efforts across HRSA, where appropriate.

• Develop and prepare a complete OMB clearance package.

• Assure reliability and validity of evaluation results, including through the development and analysis of a database used to collect, store, aggregate, summarize, and report results.

• Develop a statistical analysis plan for quantitative data and textual analysis plans for qualitative data.

• Develop recommendations and actionable strategies/methodologies that the program can use to inform future program policies and investments.

• Identify key questions to assess any environmental and policy changes that may affect the program's ability to achieve its goals and objectives.

• Develop an approach to publish evaluation findings report in cooperation with Federal staff (with authorship considerations, as appropriate). Final reports should be reviewed by an editor prior to submission. Key elements in final report review include proofreading and editing for clarity, and tailoring language to the target audience. Final report should include infographics and data visualizations to improve audience understanding of results.

• Develop an approach to submit a manuscript(s) to peer reviewed journals with authorship considerations, as appropriate. Manuscripts should be reviewed by an editor and subject matter experts prior to submission. Editorial review should include, but not be limited to, proofreading, copy-editing, and ensuring that the manuscript complies with the requirements with the peer reviewed journal. Subject matter review should include the content of key references and technical clarity.

Identify the specific journals to which the manuscripts will be submitted. Ensure that submitted manuscripts acknowledge that published work was supported with HRSA funding. If HRSA authors are included, the manuscript will need to undergo HRSA clearance.

• Implement a comprehensive review process to ensure that all products meet the established criteria for quality, accuracy, and thoroughness, including standards for 508 compliance.

Hypothetical Project #2: Policy Analysis Project

The Offeror shall prepare an analysis of a policy issue (e.g., contextual and/or policy change) related to a HRSA program that will be reauthorized within the next year. The policy issue may be one that currently exists, is anticipated in the future, or one that the Offeror thinks should be brought to the attention of policymakers.

In preparing the analysis of this policy issue, the Offeror shall assume a nine-month timeframe and shall address each of the following project requirements:

• Define the policy issue, including background and explanation of associations and/or causal relationships between the policy and the program, as well as implications of the policy for the program's purpose, program implementation, and components, awardees (recipients of HRSA funding), and program participants.

• Develop a strategy to determine/assess the national, state, and local context for the policy issue through the analysis of national, state, and local data.

• Describe which underserved HRSA populations may be affected by the policy issue and how they are likely to be affected.

• Describe an approach for identifying and engaging relevant stakeholders, including people with lived experience, on the policy and options for changes.

• Identify and provide a balanced assessment of options that policymakers could consider pursuing to address concerns associated with the policy issue.

• Develop an approach for working within limited resources (time and budget).

• Develop an approach for developing a report appropriate for publication, written in cooperation with Federal staff (with authorship considerations, as appropriate).

Reports should be reviewed by an editor and subject matter experts to ensure technical accuracy and appropriate reading level.

• Implement a comprehensive review process to ensure that all products meet the established criteria for quality, accuracy, and thoroughness, including standards for 508 compliance.

Hypothetical Project #3: Descriptive and Predictive Analytics Project

The Offeror shall develop descriptive and predictive analytics for a community-based program with a small number of grantees located in rural communities throughout the United States. The program’s goal is to improve access to behavioral health care and maternal health care in rural communities. This project will help HRSA better target its investments to support access to care.

In preparing the descriptive and predictive analytics project in response to this hypothetical program, the Offeror shall assume a two-year timeframe and shall include the identification of strategies developed to address each of the following project requirements:

• Develop a data collection and data analysis plan that describes primary data collection and incorporates secondary data sources.

• Develop a plan/approach for working with a small sample size (i.e., 12 grantees), including mitigation of data limitations, capacity and resources for extensive data collection.

• Define the project parameters, including definitions for behavioral health care providers, maternal health care providers, and rural areas/communities.

• Identify necessary data sets to be acquired and analyzed to perform descriptive and predictive analytics.

• Prepare data use agreements for federal, state, or local data sets to be acquired.

• Describe the process for securely transferring and storing data sets.

• Conduct data analysis to identify trends over the last five (5) years of access to behavioral health care and maternal health care in rural communities.

• Conduct data analysis to identify trends over the last five (5) years of the costs of delivering behavioral health care and maternal health care in rural communities.

• Conduct data analysis to identify trends over the last five (5) years of behavioral health and maternal health-related outcomes.

• Use statistical modeling to forecast future trends for access to behavioral health care and maternal health care in rural communities over the next five (5) years.

• Develop a plan for an interactive website/web page, internal to HRSA, to display past trends and future projections, enabling HRSA staff to apply filters (such as for geographic location or type of care) and modify assumptions (such as provider supply and patient demand).

• Prepare an infographic/fact sheet for distribution to a non-technical audience summarizing the key findings from the project.

• Document the methodology used for the analytics, the data analyzed, and the limitations of the project.

• Institute a comprehensive review process to ensure the quality, accuracy, and thoroughness of all products.

• Ensure the interactive website/web pages protect sensitive data and comply with federal standards, including standards for 508 compliance.

Hypothetical Project #4: Data Optimization Project

The Offeror shall develop a data optimization project to identify current data collection efforts, external data collection efforts, and a harmonized data collection approach for provider screening/testing of sexually transmitted infections, currently, an activity tracked by multiple programs within HRSA, as well as the Centers for Disease Control and Prevention (CDC), Centers for Medicare & Medicaid Services (CMS), and Office of the Assistant Secretary for Health (OASH).

In developing the data optimization project in response to this hypothetical program, the Offeror shall assume a two-year timeframe and shall include the identification of strategies developed to address each of the following project requirements:

• Identify and acquire datasets from HRSA, CDC, CMS, and OASH (through data use agreements), and/or external sources (through data purchase), to answer the following questions:

o What percentage of adolescents (by program) receive routinely recommended doses of HPV vaccine?

o What percentage of adults are screened/tested for syphilis, gonorrhea, chlamydia, and HIV?

o What are the syphilis, gonorrhea, chlamydia, and HIV rates for adults?

• Ensure secure electronic data transfer of datasets, as appropriate.

• Employ best practices including machine learning where applicable for data asset integration.

• Provide a summary report, a quality control/sensitivity report to assess performance, and all analytic code or extraction-transfer-load (ETL) procedures generated for HRSA to incorporate into its data systems.

• Provide recommendations for making HRSA-collected and HRSA-acquired data more interoperable. This may include consistent use of geographic data, unique identifiers (e.g., UEI, EIN, TIN, CCN, etc.), or data formats that are easier to manipulate and import into analytic software packages. Specifically, data interoperability measures should include identifying the key variable(s) that uniquely identify the providers or programs across each data set, and to the greatest extent possible, contain a standardized, uniform listing of provider names, street addresses, city and state names, ZIP codes, and geographic coordinates.

• Establish an internal repository of data analytic products with associated programming codes (e.g., SAS, Structured Query Language [SQL] scripts, statistical package syntax, Extraction Transfer Load [ETL] code, Juypter notebooks, R program code, Python script, etc.) available to HRSA statisticians, epidemiologists, and evaluators.

• Store datasets/files on a secure server in compliance with HHS security policies on data collection, transmission, and storage, as well as in a 508-compliant version.

Follow all HRSA, HHS, and Federal IT policies including firewalls, data encryption, two-factor authentication, privacy, and security.

• Identify gaps and inconsistencies in data collection across data sets.

• Propose a common, harmonized data collection instrument for HRSA, with specific data variables to be collected. Ensure compliance with OMB, HHS, and HRSA data collection requirements, including requirements of the Paperwork Reduction Act, Human Subjects Protection and Institutional Review Board.

• Develop a data instrument that will collect information from clients about the performance of providers in providing STI screening and testing, according to program specifications (no more than 10 questions, no protected identifiable information, unique identifier).

• Develop an instruction manual identifying the data definitions, response options, data source, and instructions on completing the questions.

• Develop training materials and deliver live and recorded trainings, using a variety of tools to include PowerPoint presentations, hands on, live demos to the data abstractors (program staff).

• Institute a comprehensive review process to ensure the quality, accuracy, and thoroughness of all products, including 508 compliance.

B. Hypothetical Factors

The Offeror shall demonstrate an understanding of, and ability to, address each of the key technical skills/capacities listed below. The Offeror shall include a specific example of recent (within the past 5 years) and relevant experience pertaining to health, public health and/or populations similar to those served by HRSA, for each individual technical skill/capacity. Individual, specific example descriptions should be limited to 300 words.

The key technical skills/capabilities related to program evaluation include the following:

• Obtaining and reviewing background information/materials on the program to be evaluated including 1) broad-based environmental scan and/or literature review on relevant/pertinent questions to be addressed; 2) interviews with key stakeholders (e.g., program staff, study program participants, other interested parties); and 3) review of data on program operations and assessment of current research on the program to be evaluated.

• Identifying appropriate evaluation design and methodology, based on evaluation purpose, available resources (time, staffing, and funding), and program size and complexity, including experimental, quasi-experimental, and non-experimental designs.

• Developing and implementing each evaluation design, methodology, and type included in Appendix A of the statement of work.

• Developing or revising data collection instruments, using plain language principles.

Offeror shall describe experience/expertise, including relevant examples of surveys and other data collection instruments.

• Preparing and submitting Office of Management and Budget (OMB) clearance packages, Institutional Review Board (IRB) Human Subjects Protection requests, and prepare documents for IRB and OMB review, as appropriate.

• Identifying appropriate sampling procedure(s) to be employed with each evaluation method (e.g., purposive, probability, and/or convenience sample).

• Collecting quantitative and/or qualitative data using appropriate strategies, including survey tools, focus groups, case studies, and semi-structured interviews, from a variety of key stakeholders, constituents, recipients, and special populations (e.g., rural, maternal/child, hard to reach populations).

• Compiling and analyzing primary, secondary, and supplementary data for addressing key evaluation questions.

• Implementing procedures such as data abstraction templates, inter-rater reliability, and measures of internal consistency among survey items to ensure consistent estimates among data collectors for the same observation(s).

• Developing instruction manuals and training materials to support data collection.

• Conducting data analyses and developing statistical models using a variety of appropriate methodologies including qualitative and quantitative methods.

• Performing statistical data analysis and developing statistical models using a wide range of analysis software, including SAS, STATA, R, SPSS, Excel, Python, Tableau, NVivo (for qualitative analyses) or other appropriate analytical and/or programming/modeling tools.

• Conducting analysis on small sample sizes to include methods such as Laplace, Convenience Sampling, and/or T Tests to get higher statistical power for small sample size data.

• Providing relevant training on the analysis of data related to evaluation studies and/or evaluation-related services.

The key technical skills/capabilities related to other evidence-building activities, including logic models/theories of change, needs assessments, evaluability assessments, organizational assessments, performance, and quality measurement, systematic reviews, policy analyses, exploratory studies, descriptive studies, and predictive analytics include the following:

• Leading in the development of a logic model to illustrate the program’s intended impact and goals; the sequence of intended effects; the correlation between activities and program effects; and where to focus outcome and process evaluations.

• Identifying a set of data points that can be used to evaluate progress and develop a specific set of performance measures or objectives, to track progress toward program goals.

• Conducting scans assessing various relevant Federal and state laws and policies (e.g., financial) and their effects on identified program(s).

• Assessing external context of identified programs and how forces in the overall health landscape (e.g., health and human service systems; social, economic, and political changes; social determinants of health; and health information technology operations and limitations) affect identified programs and their progress toward achieving goals and objectives.

• Develop a clear statement of the relevant existing policy issue(s), including the effectiveness and/or effects of policy-related strategies implemented in the field to address a specific public health issue.

• Compiling qualitative and quantitative data, previous analyses, previous policy/mission statements, previous projection analyses, and other relevant information.

• Compiling and analyzing primary, secondary, and supplementary data for addressing key evaluation questions.

• Conducting analysis on small sample sizes to include methods such as Laplace, Convenience Sampling, and/or T Tests to get higher statistical power for small sample size data.

• Applying quantitative and qualitative approaches to the analysis of policy/program alternatives using appropriate methodologies, such as case studies, survey research, statistical analysis, model building, and impact assessment.

• Assessing various policy alternatives, including implementation of various legislative requirements, that are related to identified programs or interests (i.e., how actual, planned, or potential changes in current public and private policies and implementation strategies affect the attainment of program goals and objectives).

• Determining strengths, challenges, impact, and likely future success/possible impediments of identified policy/program alternatives, on the identified program(s).

• Describing and providing a better understanding of areas for innovation and/or adapting to technological advances and the impact on health care (e.g., telehealth, mobile health systems, health IT).

The key technical skills/capabilities related to communication and dissemination include the following:

• Developing communication and dissemination plans using established approaches to scale up and sustain effective program interventions.

• Learning how interventions known to be effective can be better disseminated and facilitating the identification of key barriers, and addressing the levers that can be affected, including: 1) identifying stakeholders, target audiences, engagement roles and points of contact; 2) addressing information need(s) of the requestor including enhanced programmatic decision-making; and 3) conveying complex information in intuitive and meaningful ways for all target audience(s).

• Developing promotion and marketing strategies that leverage existing communications mechanisms (e.g., TV/radio, social media, print).

• Developing mechanisms (e.g., reports, graphical representations, dashboards) to disseminate results/findings, models of care, and evidence-based best practices among key stakeholders (e.g., program staff, recipients, other entities).

• Developing communication/dissemination resources that promote translation, implementation, and replication of evaluation studies and evidence-building findings, as well as successful models of care into improved practice for service providers.

• Seeking and incorporating feedback from internal and external entities relevant stakeholders, including people with lived experience, in support of diversity, equity, and inclusion.

• Disseminating information through a variety of mechanisms, including reports (short reports, topical reports, interim reports of findings, final reports, issue briefs, other analytical reports, including detailed methodological appendixes); data visualizations

(web-based-tools, presentations, infographics, dashboards, reports, maps); and publications (publications related to evaluation study and/or evaluation-related service methodology and results developed for peer-reviewed journals, policy briefs, abstracts for meeting/conference presentations).

• Complying with all HRSA communication policies including 508 compliance and formatting of documents, presentations etc.

The key technical skills/capabilities related to data optimization (for both quantitative and qualitative data) include the following:

• Identifying and purchasing data for analysis.

• Ensuring data quality and security, integrity, and availability of data.

• Storing datasets/files on a secure server in compliance with HHS security policies on data collection, transmission, and storage.

• Complying with all HHS and Federal IT policies including firewalls, data encryption, two-factor authentication, privacy, and security.

• Converting raw data files into summary/analytic files.

• Experience with harmonizing raw data processing/collection/storage and integration.

• Compiling, linking, aggregating, and consolidating data from multiple data sources, such as administrative data, public use files, or national data sets, into a summary/analytic file.

• Developing data documentation such as metadata and data dictionaries.

• De-identifying/anonymizing/aggregating data to ensure no personally identifiable information, public health information, or other sensitive data is released to unauthorized users, including the public.

• Conducting other related activities (e.g., data cleaning) needed to prepare data for analysis, or develop data documentation.

• Leveraging artificial intelligence, machine learning, and algorithms to support data analysis.

C. Understanding the Problem

Understanding HRSA vision, mission, goals, objectives, organization, and operation.

The Offeror shall demonstrate a general understanding of the vision, mission, goals, objectives, organization, and operation of HRSA as a public health agency of the federal government.

Understanding HRSA programs, policies, and populations served.

The Offeror shall identify and discuss no more than ten (10) major program and policy areas and related issues for HRSA priority populations around health care and public health topics that have or may have implications for services provided by HRSA areas over the next five years. These major program and policy areas shall include the key priorities identified by the Administration, by the U.S. Department of Health and Human Services (HHS), and by HRSA. For each of the major program and policy areas and related issues, the Offeror shall describe the specific implications for HRSA priority populations and programs and provide an example of how they have addressed it working with HRSA or a similar public health program.

Understanding federal requirements and standards for evaluation and evidence-building.

The Offeror shall demonstrate a working knowledge of the key Federal evaluation and evidence-building requirements (e.g., through federal legislation, OMB guidance, and HHS policy and priorities), and their potential impact/influence on HRSA's current and future evaluation studies and evidence-building services.

Understanding communication and dissemination strategies. The Offeror shall demonstrate knowledge and previous experience in developing and implementing strategies/approaches, involving development of multiple communication mechanisms (e.g., white papers; visualizations; twitter messages) to disseminate findings from evaluation studies, evidence-building activities (e.g., policy studies), and data optimization projects. These strategies shall highlight results and recommendations through a variety of mechanisms to a wide range of audience and end users, to promote knowledge and use of findings. Offeror shall include up to three (3) recent (within 5 years) and relevant examples of communication and dissemination strategies used for evaluation studies, other evidence-building activities (e.g., policy studies) and data optimization projects.

Understanding HHS and HRSA data assets.

The Offeror shall demonstrate knowledge and previous experience using HHS and HRSA data assets and systems that are relevant to conduct program evaluations and other evidence-building activities for public health and health-related programs. Offeror shall include a summary matrix of these relevant HHS and HRSA data assets and systems, indicating those assets and systems with which they have recent (within past 3-5 years) and relevant experience. Offeror shall provide up to three (3) examples of recent and relevant work for each data asset or system with which they have experience.

Example/template of the summary matrix is in Appendix Two of Section M.

In addition to the summary matrix, the Offeror shall include a brief narrative description (i.e., example) of one project for each of the data assets/systems with which they have experience. The description shall be limited to 500 words.

Understanding privacy and security issues. The Offeror shall demonstrate an understanding of the sensitive nature and importance of the information being evaluated/studied and propose appropriate measures to keep this information secure and confidential, in compliance with federal requirements.

D. Personnel

Summary of Available Personnel. The Offeror shall provide the names, educational background, professional experience (including number of years of experience), and professional/technical accomplishments of all personnel, organized by Task Area (found in the statement of work/Attachment A of the RFP) and labor category (found in Attachment B of the RFP).

The Offeror shall provide information on available personnel in a matrix format for each of the four Task Areas. Each matrix should list the individuals who may be assigned to/qualified to work on Task Order(s) resulting from this IDIQ; the listing of individuals should be organized by labor category. Individuals may appear in more than one Task Area personnel summary matrix. Example/template of matrix is included in Appendix Three of Section M. The matrix shall be presented in a font size no smaller than 10 point; it may exceed the 1-inch margin limit, if needed.

Qualifications of Personnel. Individuals leading, participating, and/or supporting activities included in this IDIQ, as well as related Requests for Task Orders (RFTOs), shall have training and experience in the areas of health sciences, public health, health services research, health care administration, economics, other relevant social sciences, health informatics, health information technology, and related aspects of program evaluation and policy analysis needed to address the various task areas included in the statement of work.

In addition, these individuals shall have training, expertise, and experience in the activities, technical skills and abilities included in the statement of work for each Task Area (Appendix A of RFP).

Resumes. The Offeror shall provide resumes for all personnel included in the Task Area personnel matrices. These resumes shall be provided in an appendix to the Offeror’s proposal and shall not exceed three (3) pages per individual. If an individual appears on more than one Task Area personnel matrix, the resume need only be included once.

The Offeror shall submit only information on education or experience that is relevant to the proposed effort. The Offeror shall delineate the specific duties of each individual as well as their expertise and skills. The Offeror shall delineate the staff competence and demonstrated experience in the skills necessary for the requirements included in the statement of work.

Subcontractors. If subcontractors are being proposed, the Offeror shall provide information to support the qualifications of the subcontractors such as resumes (or in the absence of a resume, a curriculum vitae), where appropriate and signed letters of commitment. The Offeror shall classify all subcontractor personnel in accordance with the task areas and labor categories. The Offeror shall identify subcontractors clearly in the written proposal. The Offeror shall include resumes (or in the absence of a resume, a curriculum vitae) in an appendix; individual resumes shall not exceed 3 pages.

E. Project Management, Staffing, Planning Approach

Project Management

Project Management Plan. The Offeror shall provide a detailed management, staffing, and project plan that indicates how it will support efficient and effective project management (including identifying challenges, barriers, risks, and solutions in the execution of the task order), divide organizational roles and responsibilities, make decisions, monitor work (including tasks, activities, milestones), ensure the quality of products and deliverables, and adhere to established timelines and deadlines.

Oral and…