15-233-SOL-00189Solicitation.docx

Solicitation# 15-233-SOL-00189 Page 98 of 124 Behavioral Interventions to Advance Self Sufficiency Next Generation (BIAS)

SECTION B: SUPPLIES OR SERVICES AND PRICE/COSTS

B.1 In consideration of performance of the work described in Section C, Line Items/Specifications/Statement of Work, the Contractor shall be reimbursed the Cost Plus Fixed Fee amount as follows:

CLIN 1 – Services Component 1 (Award – 15 Month Period) Estimated POP 9/2015 – 12/2016 This is a cost plus fixed fee completion type line item for non-commercial, non-severable services. The end product of value is a final memo making recommendations for the Behavioral Interventions to Advance Self-Sufficiency Next Generation Project: See Section F2 (Services Component 1 Task 10).

CLIN 2 – Optional Services Component 2 (Exercise Option – 27 month period) Estimated POP 7/2016 – 10/2018) This is a cost plus fixed fee completion type line item for non-commercial, non-severable services. The end product of value is a set of final reports on evaluation findings: See Section F2 (Optional Services Component 2 Task 6).

CLIN 3 - Optional Services Component 3 (Exercise Option – 26 month period) Estimated POP 9/2017 –11/2019 This is a cost plus fixed fee completion type line item for non-commercial, non-severable services. The end product of value is a set of final reports on evaluation findings: See Section F2 (Optional Services Component 3 Task 6).-

CLIN 4 - Optional Services Component 4 (Exercise Option – 26 month period) Estimated POP 9/2018 – 11/2020 This is a cost plus fixed fee completion type line item for non-commercial, non-severable services. The end product of value is a set of final reports on evaluation findings: See Section F2 (Optional Services Component 4 Task 6).

CLIN 5 - Optional Services Component 5 (Exercise Option – 26 month period) Estimated POP 9/2019 – 11/2021 This is a cost plus fixed fee completion type line item for non-commercial, non-severable services. The end product of value is a set of final reports on evaluation findings: See Section F2 (Optional Services Component 5 Task 6).

CLIN 6 - Optional Services Component 6 (Exercise Option – 15 month period) Estimated POP 9/2021 – 12/2022 This is a cost plus fixed fee completion type line item for non-commercial, non-severable services. The end product of value is the final report on the Behavioral Interventions to Advance Self-Sufficiency Next Generation Project: See Section F2 (Optional Services Component 6 Task 3).

* The timing of the actual exercising of optional services may vary and is at the discretion of the Government. Optional services components periods of performance may overlap.

USE OF FUNDS: Funds for each CLIN may only be used for work under that CLIN and cannot be used for any other CLIN.

B.2 Proposal Assumptions

Budget Assumptions

Offerors may use the following assumptions on which to base cost estimates:

Services Component 1:

· Task 2: As noted in the Statement of Work, the requirements of this task are not intended to be as extensive as the consultations completed under the original BIAS contract (described in the Background section of the Statement of Work). Offerors should propose a number of consultations and method for consultations considered necessary to accomplish the goals of this task as outlined in the Statement of Work. For budget purposes Offerors can assume that the consultations will not require distance travel and will take place either locally to the Contractor work site or via teleconference

· Task 4: Offerors shall assume that the primary goal of site engagement and recruitment in Services Component 1 is to identify 3-5 sites for the purposes of designing and testing behaviorally-informed interventions in Services Component 2. However, during site engagement and recruitment in Services Component 1 the Contractor may also engage with a greater number of sites than necessary for the purposes of Services Component 2, with a goal of building relationships for site engagement and recruitment over the course of the entire project.

Optional Services Component 2:

· Task 3: Offerors shall assume that the primary goal of site recruitment and selection in Optional Services Component 2 is to finalize recruitment and selection of 3-5 sites for Optional Services Component 2 and to begin site recruitment for Optional Services Component 3. However, the Contractor may also engage with a greater number of sites than necessary for the purposes of Optional Services Components 2 and 3, with a goal of building relationships for site engagement and recruitment over the course of the entire project.

· Tasks 3, 4, 5, 6: The contract calls for up to 5 sites to be selected and 3-5 sites to work with the Contractor to design and test behaviorally-informed interventions and be the subject of site-specific reports. For budget purposes, Offerors shall assume 4 behavioral design memos, 4 evaluation sites, and 4 site specific evaluation reports. However, Offerors shall assume that it may be necessary for more than 4 sites to begin preliminary behavioral diagnosis and design work in order to obtain 4 final sites.

· Task 4: Offerors shall assume that the Behavioral Design Memos are internal memos to the COR

· Task 5: Offerors shall assume that the Evaluation Plans may be made publicly available upon request and should be appropriately formatted and edited for public consumption; however Offerors shall assume that the Evaluation Plans will not be widely disseminated or posted to the ACF website and are not subject to all of the requirements for dissemination products as described in SC1 Task 8.

· Task 5: ACF is interested in innovative and creative proposals for the most appropriate types of information collection for this project. The goal of information collection in this project is to better understand the experience of program users and staff who are involved in behavioral interventions. It is not necessarily ACF’s intent that this contract will support large or extensive surveys of program participants or staff. ACF expects that the Contractor will propose creative and cost-effective ways to collect information via the most appropriate methods which may include but are not limited to focus groups, targeted surveys, qualitative interviews, etc.

· Task 6: For budget purposes, Offerors shall assume four 50 page reports (including all appendices) graphically designed to be visually appealing and accessible to a wide audience, subject to the requirements for dissemination products as described in SC1 Task 8.

Optional Services Component 3:

· Task 3: Offerors shall assume that the primary goal of site recruitment and selection in Optional Services Component 3 is to finalize recruitment and selection of 3-5 sites for Optional Services Component 3 and to begin site recruitment for Optional Services Component 4. However, the Contractor may also engage with a greater number of sites than necessary for the purposes of Optional Services Components 3 and 4, with a goal of building relationships for site engagement and recruitment over the course of the entire project.

· Tasks 3, 4, 5, 6: For budget purposes Offerors shall use similar assumptions as those described for Tasks 3, 4, 5, and 6 in Optional Services Component 2.

Optional Services Component 4:

· Task 3: Offerors shall assume that the primary goal of site recruitment and selection in Optional Services Component 4 is to finalize recruitment and selection of 3-5 sites for Optional Services Component 4 and to begin site recruitment for Optional Services Component 5.

· Tasks 3, 4, 5, 6: For budget purposes Offerors shall use similar assumptions as those described for Tasks 3, 4, 5, and 6 in Optional Services Component 2.

Optional Services Component 5:

· Task 3: Offerors shall assume a smaller level of effort for site recruitment and selection in Optional Services Component 5 than in prior Optional Services Components, with a focus on finalizing site selection of 3-5 sites for Optional Services Component 5.

· Tasks 3, 4, 5, 6: For budget purposes Offerors shall use similar assumptions as those described for Tasks 3, 4, 5, and 6 in Optional Services Component 2.

Optional Services Component 6:

· Task 3: As dissemination is the primary focus of Optional Services Component 6, the Offerors are expected to be particularly innovative, creative, and expansive in proposals for dissemination under this services component. Some examples of dissemination products may include, but are not limited to, presentations to the COR and other relevant policy officials; special topics reports or briefs; conference presentations; webinars; blogs; multimedia presentations; scholarly journal articles; or meetings or small conferences.

· Task 3: For the final project report, offerors shall assume a 75 page report (including all appendices) graphically designed to be visually appealing and accessible to a wide audience.

Estimated Level of Effort

The Government has provided an estimated level of effort for each of the Services Components as shown below. The Government estimates the following distribution by labor category (I = Senior; II = Associate; III= Intermediate; IV = Junior; V = Editorial; and VI = Clerical/Admin). However, Offerors are to present the labor mix they determine necessary to successfully complete the work.

Services Component 1

Category Labor Hours

Optional Services Component 2

Optional Services Component 3

Optional Services Component 4

Optional Services Component 5

Optional Services Component 6

SECTION C – STATEMENT OF WORK (SOW)

TITLE: BEHAVIORAL INTERVENTIONS TO ADVANCE SELF-SUFFICIENCY NEXT GENERATION

C.1 PURPOSE

The purpose of the Behavioral Interventions to Advance Self-Sufficiency Next Generation project is to continue the Administration for Children and Families’ (ACF’s) exploration of the application of behavioral economics to the programs and target populations of ACF. The project will design and test behaviorally-informed solutions to ACF program challenges using insights from behavioral science. This next generation project is intended to build on the activities and lessons of ACF’s Behavioral Interventions to Advance Self-Sufficiency (BIAS) project as well as to go beyond the work completed under BIAS in several ways including: working with additional ACF programs; testing additional types of interventions; collecting additional types of information; and creating dissemination materials targeted specifically to program administrators and operators.

C.2 BACKGROUND

The Administration for Children and Families (ACF), Office of Planning, Research and Evaluation (OPRE), has been a leader in exploring the application of insights from behavioral economics to human services programs and policies. Behavioral economics uses insights from psychology and other behavioral and social sciences to better understand how people behave and make decisions and why they often do so in violation of the predictions of neoclassical economics.

In July 2010, OPRE hosted a symposium on “Implications of Behavioral Research for Social Welfare Research and Policy.” The symposium brought together behavioral science researchers, social welfare researchers, foundation staff, human services program operators, and federal officials to explore how principles from behavioral economics could be applied to ACF’s programs and research. The symposium included lectures giving introductions to behavioral economics and to human services and panel discussions on the application of behavioral economics to key ACF program areas including TANF, early childhood, family strengthening and fatherhood, child support, and asset building. The symposium was intended to create synergies and stimulate discussion about how to incorporate behavioral insights into human services programs and research.

Following the symposium, in fall 2010 OPRE launched the Behavioral Interventions to Advance Self-Sufficiency (BIAS) project, the first major opportunity to apply a behavioral economics lens to programs that serve poor and vulnerable families in the United States. The purpose of the project is to apply behavioral insights to issues related to the design and implementation of social service programs and policies, with an ultimate goal of learning how behavioral insights can be used to improve the well-being of low-income children and families.[footnoteRef:1] [1: http://www.acf.hhs.gov/programs/opre/research/project/behavioral-interventions-to-advance-self-sufficiency.]

BIAS began with a knowledge development phase during which OPRE developed a strong base of knowledge of the existing behavioral economics literature, the needs of human services programs, and potential areas where insights from behavioral science could be used to address ACF program challenges. During the knowledge development phase:

· BIAS conducted a review of behavioral economics field experiments in eight domains including charitable giving, consumer finance, energy/environment, health, marketing, nutrition, voting, and workplace productivity, and produced a report on commonly applied behavioral interventions in these domains.

· BIAS project staff engaged in conversations with nearly 100 ACF stakeholders including federal, state, and local human services program administrators and human services and poverty researchers in order to introduce concepts from behavioral economics, learn more about challenges facing ACF programs, and explore possibilities for using insights from behavioral economics to address program challenges. This stakeholder engagement culminated in a two-day BIAS Peer Practicum that brought together policymakers, program administrators, and behavioral experts for an exploration of the application of behavioral economics to human service programs.

· BIAS project staff engaged in preliminary exploration of how insights from behavioral science could be applied to program challenges in three specific sites. BIAS worked with the Texas Office of the Attorney General’s Child Support Division to explore ways to increase the number of incarcerated noncustodial parents who apply for a modification to their child support order; worked with the Illinois Department of Human Services and one of its job search contractors to explore ways to increase TANF participants’ engagement in job search activities; and worked with the National Domestic Violence Hotline to explore ways to reduce the number of callers who reach a hold message but hang up before talking to an NDVH staffer.

· Through work with these three sites BIAS refined its approach to applying behavioral insights to human services programs and developed the specific method of behavioral diagnosis and design used in the BIAS project. Behavioral diagnosis and design is an iterative process of clearly identifying and defining the problem to be solved; diagnosing possible behavioral reasons for the problem; designing behaviorally informed interventions; and testing those interventions as rigorously as possible.

The BIAS report “Behavioral Economics and Social Policy: Designing Innovative Solutions for Programs Supported by the Administration for Children and Families[footnoteRef:2]” describes insights from the knowledge development stage of the BIAS project. The report includes an overview of behavioral economics, an introduction to behavioral diagnosis and design, and descriptions of BIAS’s work with the child support, TANF, and domestic violence programs described above. The report on twelve commonly applied behavioral interventions in other domains is attached as a technical supplement to this report. [2: https://www.acf.hhs.gov/programs/opre/resource/behavioral-economics-and-social-policy-designing-innovative-solutions-for-programs-supported-by-the-administration-for]

Following this knowledge development phase, the BIAS project has been working with select ACF programs to diagnosis program challenges using a behavioral economics lens and to design and test behaviorally-informed interventions. Projects include:

· Working with the Texas Office of the Attorney General Child Support Division on increasing the number of incarcerated noncustodial parents who apply for child support order modifications. (Report available here)[footnoteRef:3][footnoteRef:4]. [3: http://www.acf.hhs.gov/programs/opre/resource/taking-the-first-step-using-behavioral-economics-to-help-incarcerated-parents-apply-for-child-support-order-modifications] [4: References to reports and other public information about BIAS publications are hyperlinked throughout this background section. All hyperlinks go to information that can be found at http://www.acf.hhs.gov/programs/opre/research/project/behavioral-interventions-to-advance-self-sufficiency. Information about many evaluation projects that are not yet the subject of published reports can be found in the “newsletters” section of this page.

· Working with the Washington State Division of Child Support on increasing the number of incarcerated noncustodial parents who apply for child support order modifications.

· Working with the Franklin County Ohio Child Support Enforcement Agency on increasing amount and timeliness of owed payments collected. (More information available here).

· Working with the Cuyahoga County Office of Child Support Services on increasing amount and timeliness of owed payments collected and increasing payments immediately after orders are established but before wage withholding begins.

· Working with the Indiana Office of Early Childhood and Out of School Learning on reducing the number of visits required to recertify child care subsidies and increasing the number of Child Care and Development Fund (CCDF) voucher recipients who choose quality rated child care providers. (More information available here).

· Working with the Oklahoma Department of Human Services on increasing the percentage of families who renew their child care subsidies on time. (More information available here).

· Working with the Los Angeles Department of Public Social Services on increasing the percentage of previously exempt TANF participants who show up for mandatory reengagement appointments. (More information available here).

· Working with the Paycheck Plus Demonstration in New York City on increasing the number of eligible individuals who attend an informational meeting about the Paycheck Plus benefit. (More information available here).

Results from the first BIAS experiment, a test of the intervention implemented in Texas to increase applications from eligible noncustodial parents for child support order modifications, were released in September 2014. The behaviorally-informed intervention produced a highly statistically significant and policy-relevant impact at a relatively low cost and demonstrated the promise of applying behavioral economics to improve ACF program implementation and outcomes. Results from other BIAS tests are forthcoming and will be released on a rolling basis throughout 2015 and early 2016.

While the BIAS project is ongoing and results have not yet been released for many of the experiments, based on the positive results from Texas and early positive indications from the other sites, ACF continues to believe that the application of behavioral economics to human services is a promising approach for achieving improvements to the operation and outcomes of human services programs at relatively low cost. Moreover, it is clear that there is strong and continued interest from those who operate ACF programs at the state and local level in applying insights from behavioral economics to the implementation and operation of their programs.

Through this Behavioral Interventions to Advance Self-Sufficiency Next Generation Project, ACF intends to continue experimenting with the application of behavioral insights to human services programs. This project will build off of the strong foundation set by the BIAS project including the extensive knowledge development and lessons learned about applying behavioral interventions to human services programs.

ACF also intends that this Behavioral Interventions to Advance Self-Sufficiency Next Generation Project will go beyond the activities of the BIAS project in specific and important ways including:

· Working with ACF programs that were not the focus of experiments in BIAS, possibly including but not limited to programs in ACF’s Administration on Children, Youth and Families and ACF’s Office of Community Services, as well as engaging a greater number of TANF programs in this next generation project.

· Obtaining OMB Paperwork Reduction Act (PRA) Clearance to conduct more formal information collection from program staff and participants in order to better understand the experience of program users and staff who are involved in behavioral interventions.

· Developing and testing interventions that go beyond changing communications with clients and explore changing the choice architecture of program and policy design.

· Creating dissemination materials specifically aimed at practitioners in order to help ACF programs understand how to productively apply behavioral insights and design and conduct their own experiments.

ACF is also interested in exploring the desirability and feasibility of designing experiments that:

· “Get inside the black box” of interventions and provide information on the effectiveness of specific components of bundled interventions, including the impact of individual behavioral components.

· Track participants over time to understand whether short term improvements to outcomes such as program enrollment and completion based on behavioral interventions lead to longer term improvements in well-being and other outcomes of interest to ACF.

C.3 SCOPE OF WORK

The Contractor shall furnish the necessary qualified personnel, equipment, materials, supplies, services, facilities, and otherwise do all that is necessary for, or incidental to, the performance of the work set forth below. Offerors responding to this SOW may propose alternative approaches in addition to addressing the tasks as proposed by the government; justification for alternatives (e.g., cost efficiency, etc.) must be provided with any alternatives proposed. Offerors are expected to draw on their own expertise to propose the most effective approach to addressing the tasks. The Contractor is expected to work with the Contracting Officer’s Representative (COR) to refine the activities and proposed plans once contract is awarded.

C.3.1 SERVICES COMPONENT 1 (Non-Severable) (Key)

Purpose: The purpose of this services component is to conduct foundational work that will prepare ACF to undertake experimental tests of behaviorally-informed interventions.

End Product of Value: A final memo to ACF that makes recommendations for the conduct of experimental tests of behaviorally-informed interventions based on findings from the tasks completed in this services component.

Period of Performance: 9/2015 – 12/2016

Working with the COR and other staff, the Contractor shall be responsible for the following tasks:

Task 1: Project Management and Communication Subtask 1.1: Orientation Within fifteen (15) working days of the award of contract, the Contractor shall initiate communication with the Contracting Officer’s Representative (COR) and other federal staff to discuss a range of topics, including, but not necessarily limited to, clarification of ACF’s interests and objectives for the project, roles and responsibilities of researchers and other personnel involved with the project, plans for conducting work on an efficient schedule and budget, timelines for project deliverables, and challenges anticipated and potential solutions. The Contractor shall send an agenda for this initial discussion and any other relevant materials to the COR no less than five (5) working days prior to the discussion. Within one (1) week of the discussion date, the Contractor shall submit to the COR a memorandum summarizing major decisions reached and follow-up issues identified during this discussion.

Subtask 1.2: Communication Effective communication between the Contractor and the federal government, coordinated through the COR, is essential to the success of this Services Component. This may include written communications and memoranda, regular teleconferences, follow-up communications, etc. The Contractor shall propose a schedule for regular teleconferences. The purpose of these calls is to discuss current activities, obtain input and guidance from federal staff, review progress made on completion of project tasks, and identify and discuss emerging challenges. Key Contractor staff, sub-Contractor staff, and/or consultants shall participate as needed to address progress and issues for discussion.

The Contractor shall submit a proposed agenda for each teleconference at least three (3) business days in advance of the scheduled date to the COR for comment. If there are materials that federal staff will need to review prior to the call date, such materials shall be submitted at least three (3) business days in advance of the call date. The Contractor shall submit a summary of telephone discussions to the COR for review and approval no later than three (3) working days after each regular teleconference call.

Subtask 1.3: Project Management and Progress Reports In general, the Contractor shall institute procedures to ensure effective management of the entire project, including resolving problems, maintaining an on-time schedule for deliverables, coordinating the work of subcontractors and consultants, communicating with the COR and other federal staff, and working within budget and on schedule to achieve the government’s objectives. Anticipated deviations shall be reported to the COR in a timely manner.

The Contractor shall submit written monthly progress reports throughout the period of the service component, which include the following information: activities completed and progress on each task for the report month, including expenditure data by task, monthly and cumulative expenditures, and the balance remaining. The Contractor reports shall include details related to project performance. Monthly progress reports shall be submitted to the COR no later than the 15th day of each month or before submission by the Contractor of invoices related to that period of performance, whichever is earlier.

Subtask 1.4: Services Component Work Plan In order to ensure effective project management between the project director, principal investigator(s), key staff and subcontractors, and the COR, the Contractor shall submit, no later than three (3) weeks after the orientation, a services component work plan that reflects more detailed information and understanding of the activities required to meet the goals and objectives of the project and to carry out the tasks than was reflected in the Offeror’s proposal and that reflects discussion and decisions and new or clarified information obtained since award.

At a minimum, the work plan shall include a detailed discussion by task of the methods to be used and activities to be conducted to ensure the successful completion of each task; a timeline for accomplishing tasks and ensuring timely delivery of all work products; a detailed description of major work products; a discussion of staff responsibilities for specific activities or tasks; and a detailed plan for addressing and resolving concerns or problems that may emerge during the course of the project.

Within two (2) weeks of receipt of comments the Contractor shall submit a revised work plan that reflects comments received from the COR. The work plan will be considered final upon written approval from the COR. The Contractor shall not proceed with the implementation or execution of items in the work plan without the affirmative, written consent of the COR. Acceptance of the Contractor’s proposal shall not be considered assent to the Contractor’s proposed work plan. Expenses incurred absent the prior approval of the COR are subject to denial consistent with existing federal rules and regulations.

Task 2: Consult with ACF Stakeholders and Experts As described in the background section of this Statement of Work, during the knowledge development phase of the BIAS project OPRE engaged in conversations with nearly 100 ACF stakeholders including federal, state, and local human services program administrators and human services and poverty researchers in order to introduce concepts from behavioral economics, learn more about challenges facing ACF programs, and explore possibilities for using insights from behavioral economics to address program challenges. The requirements of this task are not intended to duplicate or be as exhaustive as the consultations completed under the original BIAS contract. However, in order to ensure that the behavioral interventions designed and tested in this project are of interest to ACF policymakers and relevant to policy and program stakeholders; that they take into account recent developments in the human services and behavioral economics fields; and that they are complementary to and not duplicative of other ongoing efforts related to behavioral economics and human services, the Contractor shall engage in consultations with ACF stakeholders and other relevant experts.

At a minimum the Contractor shall engage in conversations with federal ACF staff, including staff from the ACF offices mentioned in the background section of this Statement of Work (Administration on Children, Youth and Families, Office of Community Services, Office of Family Assistance). The Contractor may also propose consultations with human services program staff operating programs funded by ACF; national research experts; other federal staff, including representatives from the U.S. Social and Behavioral Sciences Team; representatives of organizations with expertise in human services programs and policy; and/or other relevant experts. Through these consultations the Contractor shall seek to develop an understanding of policy and program priorities for ACF and its grantees that may be amenable to behavioral interventions and/or that should be prioritized for testing and to get input on other issues relevant to designing and testing behaviorally informed interventions for ACF programs. Another important element of this task is seeking input from federal ACF staff and stakeholders on potential evaluation sites to contact as part of the site engagement and recruitment process (Task 4).

The Contractor may also propose to consult with behavioral science experts under this task in order to understand any relevant developments or related ongoing efforts, to seek suggestions for any behavioral interventions that should be prioritized for testing, and to solicit suggestions for how this project can best contribute to advancing the field of applied behavioral science.

Consultations may cover aspects of the project in Services Component 1 as well as future services components that have yet to be funded by ACF.

No later than six (6) weeks after the project orientation, the Contractor shall submit a memorandum to the COR that includes a list of proposed consultations and a plan for conducting the consultations. The Contractor shall submit a revised list and plan, incorporating comments received from the COR within two (2) weeks of receipt of comments. The Contractor shall be responsible for the preparation of any materials needed for consultation with experts and stakeholders. Draft materials shall be submitted to the COR two (2) weeks in advance of a planned consultation to allow for review and approval. The Contractor shall address comments received from the COR and prepare final materials no later than one (1) week prior to a planned consultation. The Contractor shall submit written summaries of substantive discussion to the COR within two (2) weeks of such discussions.

All consultations shall be completed within seven (7) months of the project orientation. Within one (1) month of the final consultation the Contractor shall submit to the COR a memo summarizing major points made by experts and stakeholders and resulting recommendations for the conduct of the project. The Contractor shall submit a revised memo, incorporating comments received from the COR within two (2) weeks of receipt of comments.

Task 3: Engage Behavioral Science Experts Ensuring that the project benefits from the expertise of behavioral science experts is critical to the success of this project. The Contractor must demonstrate to the satisfaction of the COR that the Contractor has a sufficient plan for ensuring that the project leverages behavioral science expertise. No later than six (6) weeks after project orientation the Contractor shall submit a memo to the COR that proposes a plan for ensuring that the project benefits in an ongoing way from the expertise of behavioral scientists. The plan could include but is not limited to including behavioral science experts as core members of the project team; having a standing technical working group of behavioral science experts; inviting behavioral science experts to comment on specific behavioral intervention designs and evaluation plans; or some other proposal. In addition to any proposals for engaging external experts the plan should address how the project will engage with the U.S. Social and Behavioral Sciences Team to benefit from the behavioral science expertise available within the Federal government. The plan shall indicate the tasks or activities for which the Contractor will seek input from behavioral science experts, a list of proposed individuals, including their relevant experience and expertise and the justification for each recommendation, and a timeline.

The Contractor shall submit a revised memo, incorporating comments received from the COR within two (2) weeks of receipt of comments from the COR. If consultants or technical working group members are proposed, the COR, with input from other federal staff, shall have final approval regarding the individuals. The Contractor shall seek agreement from these individuals. The Contractor may not contact any individuals regarding this task or enter into any agreements around this task prior to approval by the COR. The Contractor shall be responsible for all resources necessary for any activities involving outside expertise (e.g., honoraria payments). The Contractor shall maintain the list of proposed consultants, including brief biographical sketches and descriptions of key areas of expertise, who have been approved by the COR, as well as a list of alternatives should any of the approved experts not accept or have to halt participation in the project at any time.

The Contractor shall be responsible for the preparation of any materials needed for expert consultation. Draft materials shall be submitted to the COR two (2) weeks in advance of the scheduled consultation to allow for review and approval. The Contractor shall address comments received from the COR and prepare final materials no later than one (1) week prior to expert consultation(s). The Contractor shall submit written summaries of substantive group or individual discussions with consultants to the COR within two (2) weeks of such discussions.

Task 4: Site Engagement and Recruitment The Contractor shall effectively engage with and market the study to prospective evaluation partner sites who are interested in working with the Contractor to design and test behavioral interventions to address key program challenges. The Contractor’s efforts shall focus on those sites that are the strongest candidates for inclusion in the study. Under this task the Contractor shall

· Develop site selection criteria;

· Gather the information needed to assess potential sites according to those criteria;

· Develop any materials needed to market the study to prospective sites;

· Assess potential partner sites’ ability, appropriateness for, and willingness to participate in an evaluation; and

· Develop recommendations for potential sites to be included in the evaluation.

In developing a plan for and conducting site engagement and recruitment the Contractor shall be mindful of ACF’s goals and objectives for this project including but not limited to:

· Working with ACF programs that were not the focus of experiments in BIAS, possibly including but not limited to programs in ACF’s Administration on Children, Youth and Families and ACF’s Office of Community Services;

· Working with TANF programs;

· Developing and testing interventions that go beyond changing communications with clients and explore changing the choice architecture of program and policy design; and

· Prioritizing sites with potential sample sizes large enough to allow for analysis that “gets inside the black box” of bundled interventions and provides information on the effectiveness of specific intervention components, including the impact of individual behavioral components.

In developing a plan for and conducting site engagement and recruitment the Contractor shall also take into account any priority research questions identified by ACF experts and stakeholders in Task 2.

ACF expects that the Contractor will identify and engage with a larger number of organizations than the final number expected to be included in an evaluation in order to explore and determine those most likely to become viable evaluations sites. It is unlikely that all the sites contacted or explored will participate in the study. Based on their expertise and previous experience, the Contractor shall implement a strategy that will yield a sufficient number of intervention sites.

No later than six (6) months after the project orientation the Contractor shall submit to the COR a plan for site engagement and recruitment including proposed site selection criteria, information needed to assess potential sites according to those criteria, a preliminary list of potential sites to contact, and a plan for contacting the sites and obtaining the necessary information. The Contractor shall submit a revised plan, incorporating comments received from the COR within two (2) weeks of receipt of comments.

Upon approval of the plan the Contractor shall begin gathering information about and, as appropriate, engaging potential evaluation sites. The Contractor shall obtain COR approval prior to contacting any individuals or organizations. The Contractor shall also submit to the COR and obtain approval for any materials developed to market the study to prospective sites.

Under this services component, engagement with promising sites may include preliminary behavioral diagnosis and design activities.[footnoteRef:5] The Contractor may work with promising sites to identify and clearly define program challenges, work to diagnose program bottlenecks that may be amenable to behavioral solutions, and begin to brainstorm ideas for interventions. [5: Behavioral diagnosis and design was the approach used to apply behavioral insights to human services programs in the Behavioral Interventions to Advance Self-Sufficiency (BIAS) project. Throughout this statement of work “behavioral diagnosis and design” is used a shorthand term to refer to the process or working with sites to identify program challenges and design and test behavioral interventions. However, offerors may propose a method for this work that is different from the behavioral diagnosis and design process used in BIAS.]

As information is obtained about and from organizations the Contractor shall present the information on a regular basis to the COR both in written updates and via the regular project management calls. Materials to be discussed on project calls shall be submitted at least three (3) business days in advance of the call date, as per the requirements of Task 1 under this services component. Other written updates shall be submitted on a schedule proposed by the Contractor and approved by the COR.

Task 5: OMB Paperwork Reduction Act (PRA) Clearance A key way in which this contract will go beyond the work of the original BIAS contract is by obtaining OMB Paperwork Reduction Act (PRA) Clearance to conduct more formal information collection from program staff and participants in order to better understand the experience of program users and staff who are involved in behavioral interventions.

It is not necessarily ACF’s intent that this contract will support large or extensive surveys of program participants. ACF expects that the Contractor will propose creative and cost-effective ways to collect information via the most appropriate methods which may include but are not limited to focus groups, targeted surveys, qualitative interviews, etc.

Given that this project will involve multiple tests of different interventions in different program areas in different sites and that the tests may be developed and executed on a relatively short timeframe, the Contractor shall work with the COR to consider and propose the most appropriate and useful strategy for seeking approval for information collection. The Contractor may consider developing a core set of instruments that would apply to all sites; the Contractor may consider exploring options for getting approval for information collection for the project as a whole and getting agreement for expedited approval for specific information collection instruments for particular sites; or some other strategy. This will require careful collaboration and consultation with the COR and other federal staff.

After the Contractor, the COR, and OMB if necessary have come to agreement about the plan for approval of information collection, the Contractor shall prepare the required publication notices and clearance package for submittal to OMB.

The Contractor shall submit the proposed OMB justification package and proposed information collection instruments well in advance of the planned initiation of the information collection, allowing at least 240 days for the clearance process. The Contractor shall submit a draft OMB justification package, including 30- and 60-day Federal Register notices, to the COR for review and approval on a schedule proposed by the Contractor and approved by the COR based on the initial discussions held as part of this task. The Contractor shall submit a revised package, incorporating comments received from the COR, within two (2) weeks of receipt of comments.

ACF will submit the final package to OMB. The Contractor shall address comments received from OMB and revise the instruments as needed until approval is received from OMB. Revisions should be submitted within two (2) weeks of receipt of comments.

Since this study will also collect personally identifying and/or sensitive information, the Contractor shall seek a Certificate of Confidentiality from the National Institutes of Health (NIH). This shall be noted in the Contractor’s OMB package. ACF may require the Contractor to submit draft materials before they are submitted to NIH. These materials should be submitted on a schedule proposed by the Contractor and approved by the COR based on the initial discussions held as part of this task. Once the Contractor has obtained the Certificate, they shall provide ACF with a copy of the approval within one (1) week.

Institutional Review Board (IRB) approval is needed to ensure that human subjects are protected. As part of their proposal for the Certificate of Confidentiality, the Contractor must have approval from their firm’s IRB. The Contractor shall submit draft IRB materials to the COR for review. The schedule for submission of these materials shall be according to a schedule proposed by the Contractor and approved by the COR based on the initial discussions held as part of this task and the progress of the project. Within two (2) weeks of receipt of comments, the Contractor shall provide revised materials to the COR. The Contractor shall submit final approved materials to their IRB. The Contractor shall provide the COR with a copy of the final, approved IRB materials within one (1) week of approval.

Task 6: Data Security and Monitoring Plan The Contractor shall provide a detailed security plan for protecting any and all personally identifiable information collected in the study. The Contractor shall submit a draft data security plan to the COR no later than nine (9) months following the contract award date. The data security plan shall specify the following:

1. What personal or sensitive information is collected including content of interviews, notes for tracking purposes, personally identifiable information?

1. What assurances are provided to respondents about privacy of their information?

1. How are data stored? This should include confirmation data security includes encryption procedures and levels that meet the HHS standards.

1. How is potentially sensitive information transmitted? What personal information is included in these transmissions? Is there encryption for electronic transmission? What protections are in place for materials handled via mail services? This should include discussions of data among staff within the facility as well as between facility staff and field staff.

1. What personal information do field staff maintain on cases that are in the field? How are they or relevant program staff instructed to handle and store case folders that they are working on?

1. What training procedures are in place for interviewers and specific instructions to interviewers related to handling personal or sensitive information?

As part of the detailed data security plan, the Contractor shall describe their processes for reporting breaches of data security protocol to ACF. All data security breaches, including breaches of protocol MUST be reported to ACF no later than 24 hours after the Contractor’s staff is made aware of the breach. That is, from the moment that any member of the Contractor’s (or any subcontractor’s or consultant’s) staff is made aware of the data breach, the prime Contractor must report the breach to the COR.

The Contractor shall revise the plan based on COR feedback and shall submit a revised document within two weeks of receiving comments from the COR. The plan will be considered final upon written final approval of the COR.

Task 7: Data Archiving Plan

ACF is interested in making data and analyses supported through federal funds available to other researchers. No later than nine (9) months following the contract award date, the Contractor shall submit a draft data archiving plan that establishes procedures and parameters for all aspects of data/information collection (e.g., informed consent, data maintenance, etc.) necessary to support archiving information and data collected under the contract. Topics to be addressed in the plan include, but are not limited to: identification of appropriate data archive(s) for the subject matter, data, and analysis files; the types of data expected to be collected and methods that will support the archiving of each type; identification, justification, and procedures for handling collected data that would not be archived (e.g., proprietary measures, lower quality data, etc.); methods of data storage to support archiving; data and analyses file construction and documentation that will support data archiving and duplication of all analyses; procedures to ensure data are prepared in accordance with archive requirements (e.g., documentation requirements, level of masking, etc.); considerations of the level of restriction and access for different types of data; the schedule for archiving different types or waves of data; development of a data training plan to help researchers and other interested stakeholders use the archived data; and procedures for reviewing and updating the plan to ensure it reflects and addresses changes over time in types of data collected or other changes. Final disposition of complete data, including personal identifying information, with an archive which has appropriate levels of security shall be an option subject to approval by the COR.

The Contractor shall revise the plan based on COR feedback and shall submit a revised document within two weeks of receiving comments from the COR. The plan will be considered final upon written final approval of the COR.

Task 8: Dissemination ACF is interested in ensuring that information about the project is communicated in an ongoing fashion to the range of relevant audiences identified to be of interest in and importance to the project.

Subtask 8.1: Project Description The Contractor shall develop a brief, one or two page project description for use by the Contractor and ACF/OPRE that provides an overview of the project and communicates the purpose of the project to the public. The first draft of the project description shall be submitted to the COR no later than one (1) month after the project orientation. The Contractor shall revise the draft project description based on COR feedback and shall submit a revised document within one (1) week of receiving comments from the COR.