CCEEPRA Draft PWS.docx

DEPARTMENT OF HEALTH & HUMAN SERVICES

Administration for Children Families

GOVERNMENT CONTRACTING SERVICES

PERFORMANCE WORK STATEMENT

Administration for Children and Families

Office of Planning, Research and Evaluation

Performance Work Statement for the Child Care and Early Education Policy Research Analysis

Table of Contents

**Note: as a matter of policy, some agencies require the government to prepare the PWS, the guidance at FAR 37.602(a) stipulates that “a Performance work statement (PWS) may be prepared by the Government or result from a Statement of objectives (SOO) prepared by the Government where the offeror proposes the PWS.”

Performance Work Statement (PWS) for the Child Care and Early Education Policy Research Analysis

1.0 General

1.1 Introduction

The mission of the Administration for Children and Families (ACF) within the Department of Health and Human Services (HHS) is to foster health and well-being by providing federal leadership, partnership and resources for the compassionate and effective delivery of human services. The mission of the Office of Planning, Research, and Evaluation (OPRE) is to study the ACF programs and the children and families they serve through rigorous research and evaluation projects. The purpose of this task order is to facilitate access to up-to-date research and policy resources for various audiences interested in evaluating and improving CCEE programs and applying research evidence in designing and implementing CCEE policies.

Child Care and Development Fund (CCDF) and Need for Policy Relevant Research

The Child Care and Development Fund (CCDF) is an $8.7 billion block grant program that provides funding to states, territories, and tribes to provide access to child care services for families with children under age 13 from low-income households. The CCDBG Act of 2014 (Pub. L. 113-186) was signed into law on November 19, 2014 and reauthorized as the CCDF program for the first time since 1996. The CCDBG Act of 2014 (The Act) reauthorized the law governing CCDF and represents a historic re-envisioning of the program with an increased focus on balancing its dual purposes: promoting economic self-sufficiency for families who are low-income and supporting healthy development and school success for children. The Act is designed to meet the purposes noted above through the following expanded goals: (1) protect the health and safety of children in child care; (2) help parents make informed consumer choices and access information to inform their choices of care that meet their preferences and needs and to support child development; (3) provide equal access to stable, high quality child care for children from low-income households; and, (4) enhance the quality of child care and the early childhood workforce.

On February 28, 2024, the Office of Child Care (OCC) in the Administration for Children and Families (ACF), U.S. Department of Health and Human Services (HHS) published a Final Rule, (45 CFR Part 98), (https://www.federalregister.gov/documents/2024/03/01/2024-04139/improving-child-care-access-affordability-and-stability-in-the-child-care-and-development-fund-ccdf ) to provide more details about, and amend, CCDF policies. The Act and the Final Rule contain requirements to carry out the provisions outlined above, while giving lead agencies in states, territories, and tribes flexibility in their implementation of these provisions to address their particular circumstances and populations served. See https://www.acf.hhs.gov/occ/outreach-material/2024-ccdf-final-rule for more information.

The OCC has invested significant resources in research and evaluation to identify effective policies and practices that support the implementation of CCDF, foster the employment and economic self-sufficiency of low-income working parents, and promote high quality CCEE environments for children. Beyond this ACF led work, there are researchers across the nation examining CCEE questions of importance to states, territories, tribes, and local communities. It is critical that policy makers and researchers have full access to evidence-based products from these efforts. The work under the Child Care and Early Education Policy Research Analysis (CCEEPRA) project to be funded by the Office of Planning, Research, and Evaluation (OPRE) through this solicitation will meet this important need. OPRE’s CCEEPRA project has a longstanding reputation for identifying, synthesizing, and creating (through new data collection) policy-relevant evidence.

1.2 Objective

The purpose of the CCEEPRA project is to conduct research to answer questions identified by OPRE in the ACF that will inform the activities of programs providing CCEE services to children and families in households with low incomes, especially those eligible for, or participating in, the CCDF child care subsidy program in states, territories, and tribes, including children in underserved areas, infants and toddlers, children with disabilities, and children receiving care during nontraditional hours. Research and policy analyses activities shall occur in consultation with experts in areas of critical relevance to CCEE, including but not limited to:

a. child development;

b. economics;

c. demography;

d. CCEE cost modelling;

e. survey methodology and psychometrics;

f. measuring access to high quality CCEE;

g. policies to support parental employment, training and education;

h. initiatives to improve the quality of CCEE programs;

i. development of measures of quality of CCEE;

j. CCEE initiatives to support workforce development, well-being, recruitment and retention;

k. implementation of quality assessments;

l. non-traditional hour CCEE supply;

m. payment practices for providers; and

n. co-payment rates for families and access to CCEE.

Data collection, assessment, and analysis in CCEE policy and research may include activities such as:

· conducting original studies with quantitative and/or qualitative data collection to inform policy and practice and the development of new research priorities;

· identifying, cataloging, developing, and refining measures and instruments to improve the collection of data related to program policies and practices, and to program outcomes for CCEE providers, workforce, families, and children;

· identifying relevant sources of data and conducting secondary statistical analyses on national, administrative, and other original data sets to answer questions of relevance to ACF, such as a through n above;

· responding to rapid turnaround requests from ACF for policy-relevant analyses and presentation of relevant findings to meet the needs of policy-makers and leaders including for a program administrator (non-technical) audience;

· preparing written and other dissemination products based on the research activities conducted under this contract written in plain language and presented in a variety of formats and addressing a range of relevant policy, research, and practice audiences;

· synthesizing research evidence on key topics of interest to ACF to inform policy and practice, including a living synthesis approach for topics with complex and quickly evolving evidence;

· publishing methodological and analytic research documents to advance CCEE research and evaluation;

· disseminating findings using effective data visualization methods, such as: tables, graphs, infographics, geospatial visualizations, interactive dashboards;

· translation of research findings and issues for policy-makers and implementers of CCEE programs and related studies to investigate evidence use among these audiences; and

· convening of expert early childhood researchers, policy-makers, constituents, individuals with lived experience, and federal staff on CCEE research issues of relevance to ACF programs serving families and their children from households with low incomes to assess the relevance of findings for policy decision-making and implementation of programs and initiatives.

1.3 Scope of Work

The work under this contract will conduct research to answer questions identified by OPRE in ACF that will inform the activities of programs providing CCEE services to children and families from households with low incomes, especially those eligible for or participating in the CCDF subsidy program. The work of the contractor shall include: conducting original research studies; developing, cataloging and refining measures and instruments; conducting secondary statistical analyses on national and other original datasets; synthesizing research evidence; producing methodological and analytic research documents; translation and dissemination of research findings and issues for policy-makers and implementers of early care and education programs; and, convening of research and CCEE policy experts, stakeholders and Federal staff to assess the relevance of findings for policy decision-making and implementation of programs and initiatives.

The contractor shall furnish the necessary qualified personnel, equipment, materials, supplies, services, facilities, and otherwise do all that is necessary for, or incidental to, the performance of the work set forth below. The contractor is expected to work with the Contracting Officer’s Representative (COR) to refine the activities and proposed plans once contract is awarded.

1.4 Background

ACF's Evaluation Policy demonstrates ACF’s commitment to conducting rigorous, relevant evaluations and to using evidence from evaluations to inform policy and practice. The five principles of ACF’s Evaluation Policy are: rigor, relevance, transparency, independence, and ethics. It is expected that the work described above will be carried out in accordance with the following principles.

Rigor. ACF is committed to using the most rigorous methods that are appropriate to both the evaluation questions and the populations, circumstances, and settings that are the focus of study; and that are feasible within budget and other constraints. Rigor requires ensuring that inferences about cause and effect are well founded (internal validity); requires clarity about the populations, settings, or circumstances to which results can be generalized (external validity); and requires the use of measures that accurately capture the intended information (measurement reliability and validity). When conclusions about program efficacy or effectiveness are examined, studies should use methods that isolate to the greatest extent possible the impacts of the programs or services from other influences, such as trends over time, geographic variation, or pre-existing differences between participants and non-participants. For such causal questions, experimental approaches are preferred. When experimental approaches are not feasible, high-quality quasi-experiments offer an alternative. ACF will develop and use methods that are appropriate for understanding diverse populations, taking into account historical, contextual, and cultural factors. Where possible, evaluations will design data collections to allow disaggregation of data and analyses of sub-groups to support understanding of equity.

Relevance. It is ACF’s policy to integrate existing evidence and opportunities for further learning into all activities. Where an evidence base is lacking, ACF is committed to building evidence through strong evaluations. To be most relevant, ACF will require contractors to meaningfully engage groups and individuals from the programs and communities invested in the outcomes of studies to improve clarity of presentations, accuracy of interpretations, and effectiveness of dissemination activities; ACF partners such as states, territories, tribes, and local grant recipients; the populations served; researchers; and other groups and individuals. Further, evaluations designed, and measurement used under this task order should represent the diverse populations served by ACF. Finally, findings must be disseminated in ways that are clear, accessible, and useful to our diverse range of audiences; this includes using plain language, using inclusive language, adhering to principles of clear communication, and developing products accessible to people with disabilities. In order to ensure the findings are accessible and useful to policymakers, service providers, the communities that ACF services, and other groups and individuals.

Transparency, Data Archiving, and External Communication. ACF values transparency in research and evaluation and is committed to making information about planned, ongoing, and completed evaluations easily accessible.

First, any studies conducted under this contract must be registered with an appropriate registry (e.g., clinicaltrials.gov) prior to the initiation of data collection. Study plans, including analysis plans, should be specified in advance.

Second, ACF is interested in making evaluation data and analyses available for secondary use by other interested researchers. A thoughtful data archive plan will establish procedures and parameters, well in advance of any data collections, to support archiving information and data collected under this task order.

Finally, ACF is interested in ensuring that its research and evaluation work is communicated to a range of relevant audiences throughout the course of project activities. Ongoing and continuous activities are considered essential to a strong communication approach. Audiences that may be appropriate include, but are not limited to: policy-makers, intermediaries, researchers, advocates and opinion leaders, program administrators, training and technical assistance providers, practitioners, and the communities that ACF services. ACF will require contractors to meaningfully engage groups and individuals from the programs and communities involved in studies to improve clarity of presentations, accuracy of interpretations, and effectiveness of external communications activities. ACF is particularly interested in the creation of project external communication plans that identify specific audiences and approaches and products to communicate to those audiences. ACF is also interested in communication plans that use new and emerging media, as well as innovative approaches to communication products that are compatible with the ACF website and other communications platforms.

Independence and Active Engagement. ACF’s evaluation policy upholds the idea that independence and objectivity are core principles of evaluation. Agency and program leadership, program staff, service providers, populations and communities studied, and others participate actively in setting evaluation priorities, identifying evaluation questions, and assessing the implications of findings to help ensure the relevance of the work. While active engagement of groups and individuals who are invested in the outcomes of evaluation can support relevance as well as the accuracy of program implementation details, it is important to insulate evaluation functions from undue influence and from both the appearance and the reality of bias. To promote objectivity, ACF protects independence in the design, execution, analysis, and reporting of evaluations, as outlined in the evaluation policy.

ACF aims to be informed by the practice and empirical knowledge available to inform such topics as measurement choices, technical assistance, training and site support, and the contextualizing of findings. Meaningful involvement of groups and individuals in the evaluation may increase the rigor of the activities of the task order itself as well as understanding, acceptance, and utilization of evaluation findings. Groups and individuals who are invested in the outcomes of research or evaluation include agency and federal, state or local program leadership, program staff (both at ACF and at the grantee or other programs being evaluated), service providers, developers of innovations or practices, participants in the programs, and the communities where programs occur. As appropriate, groups and individuals should have the opportunity for input at multiple phases of a project, but this input must be accomplished in a transparent way while safeguarding the objectivity and independence of the study.

Ethics and Data Safety. ACF-sponsored evaluations are conducted in an ethical and equitable manner; safeguard the dignity, rights, safety and privacy of participants; and comply with both the spirit and the letter of relevant requirements such as regulations governing research involving human subjects. ACF will expect contractors to meaningfully engage groups and individuals from the programs and communities involved in studies to ensure programmatic, cultural, linguistic and historical nuances are accurately and respectfully addressed from the initial study design, through execution, analyses and reporting.

ACF is committed to ensuring that data collected as part of this contract are maintained and stored securely. For the purposes of this section:

Personally Identifiable Information (PII) is defined as information that can be used to distinguish or trace an individual’s identity, such as their name, social security number, biometric records, alone or when combined with other personal or identifying information, which is linked or linkable to a specific individual, such as date and place of birth, mother’s maiden name, and so on.

The Role of Active Engagement for OPRE Projects

OPRE defines active engagement as the intentional involvement of groups and individuals who are invested in the outcomes of the research or evaluation. This should occur in all relevant phases of the project. The purpose of active engagement is to improve the quality, rigor, and relevance of OPRE projects, facilitate the uptake of findings, and increase capacity to promote equity (see “Making Evaluations Rigorous and Relevant” https://www.acf.hhs.gov/opre/report/making-evaluations-rigorous-and-relevant-role-active-engagement-development-learning). OPRE seeks input from people with practice, empirical, and lived expertise to inform project activities. This includes but is not limited to refinement of conceptual models, developing research questions, measurement choices, evaluation technical assistance, training and site support, the contextualization of findings, and external communication priorities.

To this end, OPRE expects contractors to develop plans and implement strategies to maximize opportunities for active engagement in ACF projects that are appropriate to the goals of each phase of a project’s life cycle. Approaches can range from shared ownership of projects (e.g., co-creation) and community based participatory research to consultation for specific activities. Opportunities to provide input shall be considered for all phases of the project from refining research questions to external communication of findings, as appropriate for the scope and goals of the project.

Groups and individuals to actively engage may include but are not limited to service providers; developers of interventions, innovations, or practices; program participants; subject matter experts; federal, grantee, or other program staff; federal, state, or local leadership; groups and individuals in programs’ communities; individuals with lived expertise; and those in broader fields of study related to human services research and evaluation.

To promote objectivity, ACF protects independence in the design, conduct and analysis of evaluations, as outlined in the agency evaluation policy. Active engagement input shall be obtained in a transparent way while safeguarding the objectivity and independence of the study, to insulate evaluation functions from undue influence and from both the appearance and the reality of bias.

1.5 Contract Type

Cost Plus Fixed Fee contract for non-severable services for a period of a performance of 60 months.The contract will include one base period (Services Component 1 (SC1)) for 60 months (5-years) and two Optional Services Components (OSCs) for 24 months to be exercised during the 60 months of the base period. OSCs, will be exercised once planning and design of the work to be conducted is completed under SC1 and it is deemed to be of benefit to the government.

1.6 Period of Performance

The period of performance for this contract is 60 months (August 2025 – August 2030), unless the period is extended by modification to this contract. The optional services components may be elected at any time during the period of performance as required to meet study objectives, and the optional services component periods of performance are expected to overlap in time. A possible schedule of election is presented below but it does not constrain the government’s ability to elect optional services sooner or later, as needed.

Possible Election Dates and Period of Performance for each Optional Services Component SC1 Period of Performance: 60 months Possible Election Date: August 2025 OSC2 Period of Performance: 24 months Possible Election Date: August 2028 OSC3 Period of Performance: 24 months Possible Election Date: December 2028

1.7 Place of Performance

The work to be performed under this task order will be performed at the Contractor’s address and may include multiple states and localities.

1.8 Recognized Holidays

Juneteenth

In addition to the days designated as holidays, the Government observes the following days:

· Any other day designated by Federal Statute

· Any other day designated by Executive Order

· Presidential Inauguration Day

· Any other day designated by the President’s Proclamation

It is understood and agreed between the Government and the Contractor that observance of such days by Government personnel shall not otherwise be a reason for an additional period of performance or compensation entitlement except as set forth within the contract. If the Contractor’s personnel work during the holiday, the Contractor may reimburse them. However, no form of holiday or other premium compensation will be reimbursed as a direct or indirect cost other than their regular compensation for the time worked.

1.9 Applicable Documents

The following documents are essential for performance of the contract:

· Guidance for OPRE Product Preparation Review and Communication

· Federal Plain Language Guidelines

· OPRE Clear Communications Checklist

· OPRE Document Review Cover Page Template

· Quality Assurance Surveillance Plan (QASP)

2.0 Specific Requirements/Tasks

Task 1: Project Management and Team Communication Effective communication between the Contractor and the federal government, coordinated through the Contracting Officer’s Representative (COR), and successful management of the project as a whole are essential to the success of this Service Component, and the project more broadly.

Subtask 1.1 Orientation The need for communication begins at the very start of the contract. Within five (5) working days of the award of contract, the Contractor shall initiate communication with the COR and other federal staff to discuss a range of topics, including, but not necessarily limited to, clarification of ACF’s interests and objectives for the project and planning for the web-based orientation convening to be held no later than thirty (30) days after award of contract. The orientation convening shall cover in-depth discussion about the goals of the contract, conducting work on an efficient schedule and budget, coordination with other ACF program offices, methodologies to be employed, timelines for early project deliverables, and the challenges anticipated and potential solutions. The Contractor shall send an agenda and any relevant materials to the COR no less than seven (7) working days prior to the discussion. Within two (2) weeks after the discussion date, the Contractor shall submit to the COR a memorandum summarizing major decisions reached and follow-up issues identified during this discussion.

During the Orientation discussion, ACF OCIO staff will provide an overview of the ACF OCIO Intake Request Form used to formalize/start the ACF OCIO Governance Framework requirements. The Contractor shall complete an ACF OCIO Intake Request Form within five (5) days after the discussion and submit it to the ACF Tech Customer Engagement Team, customer_engagement@acf.hhs.gov.

Subtask 1.2 Ongoing communication Ongoing communication is also necessary to ensure successful completion of this service component, and should be both verbal (e.g., regularly scheduled conference calls) and written (e.g., monthly progress reports). The Contractor shall propose a schedule for biweekly telephone/web-based meetings to update federal staff on progress and issues. Key Contractor staff, sub-Contractor staff, and/or consultants shall participate as needed to address progress and issues for discussion. The schedule for telephone/web-based meetings may be changed as agreed upon by the COR. The Contractor shall submit a proposed agenda for each call at least three (3) working days in advance of the scheduled date to the COR for comment. If there are materials that federal staff will need to review prior to the call date, such materials shall be submitted at least five (5) business days in advance of the call date. The Contractor shall submit a summary of meetings to the COR for review and approval no later than three (3) working days after each regular meeting.

Given the objectives of this contract and associated nature of this work, there is high likelihood that several distinct research and evaluation activities will be in progress at any given time with varying topics, methodologies, timelines, and estimated budgets. In consultation with the COR, the Contractor shall develop systems of tracking individual activities that are in progress that communicate status updates, next steps, and targeted timelines. These tracking systems (i.e., activities trackers) will be submitted to the COR on a regular schedule determined in consultation with the COR.

Subtask 1.3 Monthly progress reports and invoices In addition to the ongoing calls/web-based discussions, the Contractor shall submit written monthly progress reports throughout the period of the service component, which includes the following information: work completed and progress on each task for the report month, including expenditure data by task and by individual research and evaluation activity, monthly and cumulative expenditures, and the balance remaining (i.e., variance report). All IT-related expenses shall be highlighted for ease of reporting. The Contractor reports shall include details related to project performance. Progress reports shall be submitted to the COR monthly, on a regular schedule proposed by the Contractor and approved by the COR and before or concurrently with submission by the Contractor of invoices related to that period of performance.

Subtask 1.4 Biannual full project meeting In order to take stock of progress and for planning of activities based on findings from current services components, the Contractor shall meet with the COR and other staff in ACF biannually (i.e., twice a year), to assess the work completed and to identify priorities for work to be conducted under Services Components through the period of performance of the task order contract. The Contractor shall meet with the COR and other staff via a web-based meeting platform unless an in-person convening is requested by the COR. The meeting will serve to take stock of work completed and to plan for future work. The biannual meeting will last 6 to 8 hours and shall include key contract staff and any expert consultants or subcontractors significantly involved in the work of the contract. The Contractor shall submit an agenda for the biannual meeting and any materials for discussion no later than two (2) weeks prior to the meeting date. The Contractor shall submit meeting notes of the discussions and decisions made during the meeting no later than two (2) weeks after the meeting.

Task 2: Plan and Implement Active Engagement Approach For the current study, the contractor shall develop an approach, a plan, and implementation strategies that maximize opportunities for active engagement that are appropriate to the goals of each phase of this project’s life cycle.

Subtask 2.1 Active Engagement Plan The Contractor shall provide a plan for active engagement of experts, key constituents, partners, and/or groups or individuals from the programs and communities invested in the outcomes of this work in project tasks and deliverables. To accomplish the goals of including meaningful and appropriate active engagement while upholding the principle of independence in ACF sponsored evaluations, the plan should begin with a clearly articulated rationale for the Contractor’s approach to active engagement to guide the entire project and achieve its purpose. This rationale shall guide engagement across all tasks and any options. The approach to active engagement shall describe the reason for inclusion of the individuals and groups proposed to be engaged, how and when they will be engaged, and how input will be addressed and incorporated to achieve the purpose of the study.

The plan should follow from this approach and:

· delineate the project tasks and activities where active engagement will occur;

· demonstrate how active engagement in these tasks and activities will meet specific goals of this project;

· identify potential individuals and groups who may be involved in the active engagement and indicate why or to what end they will be engaged, how they will be engaged, and the timeline for when the project plans to engage them;

· describe the budget considerations associated with the proposed active engagement; and

· delineate how the plans for active engagement will be refined throughout the project.

For the current study, at a minimum, the following tasks should include active engagement in [insert specific tasks and subtasks here]. Additionally, the plan should include explicit discussion of how the independence of research and evaluation functions and findings will be protected from both the appearance and reality of bias throughout the planning for and execution of active engagement. The Contractor shall submit a draft plan for active engagement to the COR no later than two (2) months after contract award and a final deliverable to the COR no later than three (3) months after contract award. If appropriate, a Memorandum of Understanding with some individuals and groups may be warranted, particularly in high-stakes evaluations or sensitive relationships. The Contractor’s plan shall be updated at least annually and throughout the project as necessary when key groups and/or individuals are identified to be engaged in any of the activities under the Optional Services Components under this contract to document the decisions and agreements made around areas of active engagement.

Subtask 2.2 Implementation of Active Engagement Activities The Contractor shall implement all activities in the COR-approved Active Engagement Plan in Subtask 2.1. The Contractor shall submit any draft and final deliverables in the Active Engagement Plan to the COR for review and approval on a schedule proposed by the Contractor and approved by the COR.

Task 3: Establishment of Panel of Research and Policy Expert Consultants and Convening of Experts

Subtask 3.1: Establishment of Panel of Research and Policy Experts and Engagement Plan Within two (2) months after award of contract, the Contractor shall prepare and submit a list of policy and research experts from which to select a panel that will provide expert consultation to the project and who, when appropriate and necessary, will enter in a contractual agreement with the Contractor to conduct one or more research and analyses activities under this contract, under the supervision of the Project Director(s). The contractor shall include in this list experts identified by the COR and other key staff in ACF, and discuss with the COR the combination of expertise necessary to address the research questions identified by the Agency and provide the best service to the Agency. The Contractor shall submit a final list of recommended experts and, in consultation with the COR, make the final selection of the pool of experts. Individuals in the pool of experts will collaborate with and/or serve the project in different capacities. Some experts may participate as members of one or more topical workgroups convened by OPRE for the purpose of exchanging research and policy expertise and identifying questions in need of research and resources that need to be developed to translate research evidence to inform policy and practice. Participation by individual experts in this capacity is voluntary and is intended to benefit both the experts and the activities of the contract. Other experts may be identified by the project to participate in the development of one or more products, or to contribute in other ways to key activities supported through this contract (e.g., new data collection to answer the questions identified by the Agency). In this case, these experts shall enter into a contractual or consultation agreement with the contractor in order to support their contribution to the project activities.

The Contractor shall prepare an expert engagement plan that details how the experts will be engaged to provide feedback on the different activities under this contract. The plan shall connect experts’ unique expertise, the activities they will be involved in, and the rationale for their involvement. The Contractor, with previous approval from the COR, shall establish roles and responsibilities of individual experts selected for the completion of products and/or other types of contribution to proposed activities, including time commitment and level of effort based on the activities for which their expertise is needed in the project. The contractor shall prepare a draft consultant agreement to be used for contracting the services of the selected expert(s) and submit to the COR for review and approval prior to entering in any contractual agreement. The Contractor shall submit the list of experts and the draft plan to COR no later than two (2) months after award of SC 1. COR will provide feedback within two (2) weeks of receipt of the draft. The Contractor shall submit a revised plan no later than two (2) weeks after receipt of comments from COR. The expert engagement plan shall be revised annually and/or after experts have been identified for participation in any of the Optional Services Components under this contract.

The Contractor shall contact experts selected to contribute to the activities of the contract and invite them to be part of the project. The Contractor shall send experts who agree to contribute to the deliverables from this project the agreement to contract for their services and a letter explaining their roles and level of participation in activities related to the Contract.

The Contractor shall prepare a contact list of all experts, including affiliation and contact information, and submit the contact list to the COR. The Contractor shall send updates of the list of experts participating in the activities of the project four times a year, specifying the status of their agreement with the project and the status of the work they have been contracted to perform.

Subtask 3.2: Convening of Research and Policy Experts

The Contractor shall convene the expert(s) under the following conditions:

The Contractor shall convene experts who participate voluntarily in topical workgroups sponsored by OPRE via audio calls, webinars or other electronic media at least four times a year for the purpose of sharing expertise on focal topics identified by the members of the workgroup(s) under the leadership of contractor staff, the COR and/or other agency staff. The Contractor shall schedule the workgroup convenings, develop agendas, and provide the technology necessary for convening of the experts as per approved plan under Subtask 3.1 above. The Contractor shall lead the convening of experts through the workgroups, take notes of the discussions, and submit summaries of the topics discussed to the COR and the experts participating in the workgroup convenings no later than two (2) weeks after the discussion date. Information from the discussions of experts through the workgroups will inform activities under this contract, including but not limited to: products to translate research evidence for policy and practice decision-making; development and testing of new research measures; syntheses of research findings; and, other key deliverables and activities under this contract.

The Contractor shall convene (with prior approval from the COR), experts selected to contribute to products and/or other activities under this contract, at those times deemed necessary to obtain their expert advice and/or develop and implement plans to conduct the activities for which they were selected to initiate a consultant agreement or subcontract. The number of experts to contribute to products and/or other activities will be determined by the scope of expertise needed and the level of effort required to answer the questions of relevance to the Agency. Convening of expert(s) may occur through audio and video calls, webinars, and other electronic means. If in-person discussions are required, with prior approval from the COR the Contractor shall coordinate with other Agency contractors in charge of logistics to arrange for travel and lodging arrangements for non-local expert(s) to meet with the Contractor, the COR and other key federal staff in Washington, D.C., and will pay honoraria for expert consultation services and/or time and effort for planning for products or activities.

Task 4: Policy and/or Research Assessment and Analysis, Data Collection, Syntheses, and Dissemination of Findings The topics selected for policy and/or research assessment and analyses, syntheses and dissemination of findings will be informed by ACF and selected through consultation with experts and/or as a result of direct requests from the COR. The activities to be conducted under this task will be determined by the focus of the research and assessment and the availability of existing data to conduct such assessment and analyses. In instances when required data is not available for analyses, the Contractor may be required to conduct original data collection within the parameters of the project scope approved by the COR (e.g., qualitative data collection may be needed to explore a new topic; quantitative data collection may be needed to assess the prevalence of a particular outcome, policy or event under study).

Subtask 4.1: Develop Conceptual Framework(s) and Work Plan(s)

The Contractor shall meet with the COR and any expert consultant selected for this task, to gain a broader understanding of the research and/or policy issue(s) in need of study, the policies and/or research to be reviewed and summarized, as well as the objectives of the task(s), task requirements and parameters. The Contractor, in consultation with, and with prior approval from COR: (1) shall select the issue(s) in need of research and analysis; (2) shall identify and explore the types and sources of information available for assessment, analysis and review, or the types of primary data collection needed to answer the selected question(s); and (3) assess through limited review, the potential relevance and value of each issue. The Contractor shall conduct initial review of the research evidence on the key policy and program issue(s) identified by the Contractor, the COR and the expert consultant(s). The Contractor shall develop a conceptual framework of the area(s) in need of research and analysis to guide the work to answer the key questions of interest. The Contractor shall conduct preliminary discussions with relevant and appropriate sources to identify data to be examined, or shall identify the sources of data to be included in a new collection. No later than thirty (30) days after the consultation with the COR and identification of the issues in need of research and analyses, the Contractor shall submit to the COR for review a memo describing the problem to be addressed and how the Contractor proposes to answer the key questions. The memo must summarize the research evidence gathered on the key research questions to be addressed. In addition, the Contractor shall submit a draft Conceptual Framework and a list of possible data sources to be analyzed to answer the key questions of interest, no later than two (2) weeks after approval of the memo by COR. The Contractor shall submit a final memo, conceptual framework and list of data sources no later than two (2) weeks after receipt of comments from COR.

The Contractor shall submit a work plan and schedule of deliverables that reflect the memo and related materials approved by the COR, no later than four (4) weeks after approval of the final memo and related materials. The work plan must define and conceptualize the specific policy, program and research questions and issue(s) under consideration. In this work plan the Contractor shall: (1) identify the nature and sources of data relevant to the tasks to be completed, additional data needs, sources of needed data; (2) describe data collection strategies, analytical needs, and strategies necessary to effectively answer the question(s) of relevance; (3) specify the approach to conducting the policy and research assessments and analyses; and, (4) incorporate a dissemination plan for reporting of findings from the work under this task that will meet the overarching goals of dissemination for this contract. The Contractor shall identify resources (staff, money, time, etc.) needed to execute the plan and any potential problems. The Contractor shall submit a final work plan no later than two (2) weeks after receipt of comments and requests for revisions from the COR.

The Contractor shall meet with the COR, and, if required, with expert consultant(s) to discuss the proposed work plan. If so required, the Contractor shall incorporate any revisions to the plan requested by the COR and re-submit a revised plan for final approval by COR.

Subtask 4.2: Research Design and Methodology and Data Analyses Plan

The Contractor shall submit a fully specified research study design and methodology. The Contractor shall submit a draft study design in the form of multiple memos to the COR. For the sake of efficiency, the memos should align as much as possible with the format and content required for an OMB clearance package (if required). The research study design will include, but is not limited to:

a. research questions to be answered;

b. research design that will be used to answer these questions;

c. data elements to be collected and/or analyzed;

d. if new data collection needed, proposed data collection methods (e.g., site visits, observations, telephone interviews, focus groups, web-based surveys, etc.);

e. proposed sampling approach and recruitment strategy of respondents and key informants;

f. proposed data collection measures and instruments, including already existing measures/instruments, as well as an approach to measurement development for constructs for which existing measures are not available;

g. proposed plan to pilot instruments and study protocols, if deemed necessary;

h. proposed schedule and logistics for data collection;

i. proposed management plan of data collection (e.g., includes a process for managing and overseeing data collection that will identify and address, as quickly as possible, any problems that could arise);

j. proposed approach for selecting, recruiting, and training data collectors;

k. data management plan (e.g., data storage security and ongoing security monitoring; process for cleaning of data);

l. data analysis plan;

m. data disposition plan; and

n. proposed schedule for reporting progress, completing analyses, production and delivery of data tables, reports, briefs and other materials resulting from data collection.

The Contractor is encouraged to use the most appropriate methods and measurement available to address the research objectives and questions that are the primary focus of the study. The Contractor shall assess the utility and appropriateness of existing measures and only consider developing measures if there are gaps in existing measures as they relate to the study research questions. Piloting of instruments and study protocols will be conducted to the extent necessary, in compliance with the Paperwork Reduction Act.

As appropriate, the analysis plan should include a discussion of whether frequentist methods, Bayesian methods, or a mix of both is most appropriate to address the research question(s), and explain the reasoning behind the recommended approach.

If appropriate, Offerors are encouraged to consider data visualization as an exploratory tool during analysis. Offerors should consider using state-of-the-art methods and tools, including but not limited to, social network analysis, geomapping, and/or other appropriate techniques for mapping and presenting relevant information.

The Contractor shall submit components (a) through (h) of the study design as a memo one (1) month after the work plan and schedule are finalized and approved (Subtask 4.1, above).

Components (i) through (n) will follow and be submitted as a separate memo after approval of components (a) through (h), and no later than two (2) weeks after approval by COR. The Contractor shall incorporate feedback from active engagement of experts, key constituents, partners, and/or groups or individuals from the programs and communities invested in this work, as well as the COR, prior to submission of final study design memos, and no later than two (2) weeks after approval of memos.

Subtask 4.3: Development of Data Collection Instruments

If primary data collection is needed, the Contractor shall prepare draft data collection instruments one (1) month after, and based on, the approval of the data collection study design memo (Subtask 4.2). The Contractor shall submit revisions to the draft data collection instruments based on feedback from the COR within three (3) weeks of receiving comments.

Generic OMB Clearance for Pre-testing or Piloting Instruments If needed and approved by the COR, the Contractor shall seek generic OMB approval and conduct pretesting or piloting work to ensure the feasibility and appropriate use of all data collection instruments and protocols as described in the study design memo (Subtask 4.2), including any translated versions for participants who speak languages other than English (if applicable). The Contractor shall then incorporate the findings and any needed changes to the instrument package and protocols. The revised instrument package shall be completed as per the approved study design plan approved.

If clearance related to OPRE’s Generic Clearance (e.g, Formative Data Collections for ACF Research, Formative Data Collections for Program Support, Pre-testing of Evaluation Data Collection Activities, Generic Clearance for Engagement Activities, or FastTrack Clearance for small scale feedback-related information collections) is required as per the approved study design plan, the Contractor shall allow at least 30-45 days for the clearance process from the time of OPRE approval of Generic Clearance package. Federal Register notices are not required.

For generic clearance packages, the Contractor shall submit a draft OMB justification package (including data collection protocols, final instruments, sampling plan, and any other materials to be utilized during the pilot testing of instruments and protocols) to the COR(s) for review and approval no later than three (3) months prior to when data collection is expected to begin.

The Generic OMB Clearance requires the submission of Supporting Statement A and Supporting Statement B, in addition to finalized instruments. The Contractor shall work on the two supporting statements while the instruments are being finalized. The final OMB package shall be submitted to the COR no later than two (2) weeks following receiving comments from the COR.

ACF will submit the final package to OMB. The Contractor shall address comments received from OMB and revise the instruments as needed until approval is received from OMB.

Subtask 4.4: OMB Clearance for Primary Data Collection If primary data collection is required to answer the focal questions approved in the research study design and methodology and analyses plan under Subtask 4.2 above, the Contractor shall prepare the required Federal Register Notices (FRNs) and information collection request materials for submission to OMB well in advance of the planned initiation of the work, allowing at least 240 days for the clearance process (to include ACF review through OMB approval). The timing of this package may be discussed during the initial discussion of research activities under Task 4, and all draft and final deliverables will meet the timeline approved in the work plan under Subtask 4.1, above. The Contractor shall submit draft instruments and 30- and 60-day FRNs to the COR for review and approval. After the Contractor makes any changes recommended by the COR, ACF will submit the FRNs and instruments to ACF and HHS leadership for review. The Contractor shall submit a draft OMB justification package to the COR for review and approval. After the Contractor makes any changes recommended by the COR, ACF will submit the final package to OMB. The Contractor shall address comments received from OMB and revise materials as needed until approval is received from OMB.

Institutional Review…