Statement_of_Objectives_-_00431.docx

STATEMENT OF OBJECTIVES

Domestic Violence Housing First Demonstration Evaluation

PURPOSE

This contract with the Department of Health and Human Services (HHS) Office of the Assistant Secretary of Planning and Evaluation (ASPE) is to conduct a rigorous evaluation of the Domestic Violence Housing First (DVHF) demonstration program coordinated by the Washington State Coalition against Domestic Violence (WSCADV). This evaluation will build on research conducted during the pilot phase of DVHF, building evidence on the impacts of the program by examining housing stability, survivor safety, and survivor’s and children’s wellbeing over time.

BACKGROUND

The Domestic Violence Housing First (DVHF) program was designed to provide stable housing to survivors of domestic violence as quickly as possible. Housing First models stem from efforts to provide homeless individuals and families permanent housing quickly, while providing additional services as needed. The model’s guiding theory is that individuals’ basic needs, such as shelter, must be met before they can work toward rebuilding their lives.[footnoteRef:1] The DVHF program uses this approach to address housing needs for survivors and their children, reducing the risk of family homelessness, while providing survivor-driven services focused on safety and wellbeing. DVHF draws on previous research showing positive results for programs that provided domestic violence (DV) survivors with mobile advocacy and/or housing assistance.[footnoteRef:2] DVHF’s four program pillars are: [1: Fact Sheet: Housing First. (April 2016). National Alliance to End Homelessness. www.endhomelessness.org] [2: Sullivan, C.M. & Bybee, D.I. (1999). Reducing violence using community-based advocacy for women with abusive partners. Journal of Consulting and Clinical Psychology, 67, 43-53. Baker, C., Billhardt, K. A., Warren, J., Rollins, C., and Glass, N. E. 2010. Domestic violence, housing instability, and homelessness: A review of housing policies and program practices for meeting the needs of survivors. Aggression and Violent Behavior 15:430–439. Sullivan, C.M., Bybee, D.I., & Allen, N.E. (2002). Findings from a community-based program for battered women and their children. Journal of Interpersonal Violence, 17, 915-936.]

· Survivor-driven advocacy: Advocates focus on addressing the needs identified by survivors. Advocates are mobile, meeting survivors where it is safe and convenient for them.

· Community engagement: Advocates provide outreach and education to landlords, law enforcement, city government, and housing councils on the dynamics of domestic violence and survivors’ needs for safety. By doing so, they change and improve the way communities respond to domestic violence.

· Housing stability: Like safety planning, housing stability is integrated into advocacy. Advocates work directly with survivors on accessing and/or retaining their housing, including accompanying survivors to housing appointments, acting as liaisons with landlords, and negotiating leases.

· Flexible financial assistance: Funds are targeted to support survivors so they can rebuild their lives, including covering childcare costs, transportation, school supplies, uniforms and permits required for employment, as well as time-limited and flexible rental assistance.[footnoteRef:3] [3: http://wscadv.org/resources/what-is-dv-housing-first/]

Pilot Phase.

The program was piloted with four domestic violence agencies (Cohort 1) and then expanded to nine additional agencies (Cohort 2) serving survivors with significant barriers to housing, such as those in immigrant communities, Native American communities, or survivors living in rural areas.[footnoteRef:4] Cohort 2 agencies served 681 survivors and approximately 1,000 children between September 2011 and September 2014. An evaluation of the pilot focused on documenting outcomes from intake data on 681 survivors and final follow up data on 64%, face-to-face interviews with 170 survivors, 139 survivor feedback surveys, and 36 DV agency staff focus groups. Survivors served by DVHF had increased housing stability, often without substantial financial assistance, and a very high percentage (91-96%) of survivors who received services for 6 months or longer retained permanent housing during that time. Qualitative data from the evaluation suggest DVHF is particularly promising in providing culturally responsive services and addressing the needs around family safety and stability, contributing to children’s wellbeing. [4: Mbilinyi, L., (2015). The Washington State Domestic Violence Housing First Program: Cohort 2 Agencies Final Evaluation Report: September 2011-September 2014. Prepared for Washington State Coalition Against Domestic Violence. ]

The evaluation team notes that the program was evolving over the pilot phase and the evaluation design necessarily changed too. These exploratory findings are further limited by the need to use agency staff to report participant data and survivor attrition—36% of survivors were not able to be located or declined to participate in the final follow-up.[footnoteRef:5] [5: Ibid ]

Current Demonstration.

Given the promising findings from the five years of piloting, WSCADV received foundation funding for a five-year DVHF demonstration (2015-2019). The goal of the demonstration is to: (1) improve the health, safety, stability, and well-being for domestic violence survivors and their children; (2) increase the level of evidence that documents the relationship between DV advocacy, housing stability, and improved health and well-being for survivors and their children; and (3) improve housing systems and policies so that the needs of DV survivors and their children are reflected in the housing systems of care in two selected regions of Washington State.[footnoteRef:6] Four DV agencies within Urban King County and rural South Central Washington used 2015-2016 funding to shift their DV service delivery to the DVHF model. Data infrastructure has been developed and key data collection techniques are being piloted, but a full scale evaluation of the demonstration has not been funded. [6: WSCADV (June 2015). Domestic Violence Housing First: Demonstration & Research Project Overview. http://wscadv.org/]

Current funds have allowed WSCADV and its research team to put in place basic data infrastructure to carefully track services provided to survivors and for piloting of collection of implementation data in one site. A major component of this work is building trust among service providers and assessing the feasibility of their participation in a rigorous evaluation. Under this contract, the contractor shall build on this infrastructure to design and implement an impact evaluation of the demonstration.

SCOPE

The DVHF demonstration provides an opportunity to significantly increase the evidence documenting the complex interrelationships among domestic violence advocacy, housing stability, and improved health and well-being for survivors and their children. ASPE in partnership with the HHS Administration for Children and Families (ACF) Family Violence Prevention and Services Program (FVPSA) and Department of Justice’s Office of Victims of Crime (OVC) seek to capitalize on this opportunity by awarding a contract to conduct a rigorous study that can examine DVHF impacts over time for key subgroups.

OBJECTIVES

(1) The contractor shall evaluate the DVHF demonstration project using a rigorous research design to determine housing, safety, and child well-being impacts.

(2) The evaluation shall assess subgroup impacts.

(3) The evaluation shall incorporate key implementation data on program component delivery and key contextual factors related to program delivery.

The government proposes the following research questions as the focus of the evaluation, however the offeror shall refine in their proposal based on what is known about data availability; program size and feasibility of subgroup analysis; and tradeoffs between data quality, usefulness of evidence generated for policy and practice, and resources available for this study.

Research questions:

(1) What program components, or combination of program components, increase survivors’ housing stability and safety? Do these differ by subgroups?

(2) What are DVHF impacts on survivor and child wellbeing?

(3) What components of DVHF did survivors and their children receive over time? How did these services correspond with their needs?

(4) What program, agency, and community characteristics were associated with DVHF impacts? What conditions were necessary for success?

TASKS

The offeror should submit a technical proposal that details a work plan and schedule of deliverables that accomplishes the statement of objectives and fully meets the Government’s objectives of conducting an implementation and impact evaluation of the DVHF demonstration and addresses all research questions outlined above, as modified in the contractor’s proposal. A basic task structure is outlined below, however the task structure should be modified to conform to the contractor’s proposed work plan, including indicating where preliminary work that would support this task has already been completed. The offeror is encouraged to propose innovative methods and strategies that can be accomplished within available resources and meet the standards of rigor necessary to determine causality in assessing demonstration impacts. For all deliverables the offeror shall propose a timeline that allows for two to three drafts of deliverables. Each revision shall incorporate feedback provided by the COR and/or technical expert panel when appropriate.

TASK 1: Project Management Orientation Meeting.

Within two weeks of contract award, the Contractor shall meet with HHS in Washington, DC (or via conference call) for an orientation meeting. The Contractor, in consultation with the COR, shall prepare the agenda for this meeting. A draft meeting agenda shall be delivered to the COR within 3 business days of contract award. The purpose of the meeting will be to:

· Review project goals and objectives and clarify research questions and priorities.

· Provide an overview of the evaluation approach.

· Review tasks and the deliverables schedule, including possible challenges and approaches to resolving them.

· Establish:

· Protocols for all project communications,

· Requirements for substantive and financial reporting,

· Procedures for approving deliverables, and

· Methods for handling unanticipated events.

· Address any other contract management topics.

Within two weeks of the orientation meeting, the Contractor shall provide the COR with a summary of the meeting describing each issue discussed and the conclusions reached. The Contractor shall also make note of any issues that remain and plans developed to resolve the outstanding issues.

Ongoing Project Communication.

The contractor shall submit monthly progress reports describing, by task, the work accomplished during the previous month. At a minimum, such reports shall cover the following items:

· Discussion of the activities accomplished by task/subtask since the last report;

· Activities anticipated during the upcoming reporting period;

· Difficulties encountered and remedial action(s) taken;

· Progress on data collection activities, when applicable;

· Early warning of any potential substantive or contractual issues the contractor is monitoring;

· Hours by key personnel for each task; and

· Proposed personnel and other management changes.

Monthly reports shall not exceed 5 pages and are due on the 15th of every month following the orientation meeting and shall be sent by the Principal Investigator directly to the COR via electronic mail. In addition to these written monthly reports, it is expected that the contractor shall work closely with the COR and other agency partners during the course of this project primarily via email and telephone. It is anticipated that regular conference calls will be required throughout the project.

Federal Technical Expert Panel Meetings.

HHS shall convene a federal technical expert panel of key staff from partner agencies to provide feedback and guidance at key points in the project. Three times during the performance period (e.g., to review the study design, interim findings, and final report), the contractor shall travel to Washington, D.C. for an in-person meeting with the technical expert panel. The COR, in consultation with the contractor, shall schedule meeting times and dates. The COR will reserve meeting space on government property. The Contractor, in consultation with the COR, shall prepare the agenda for this meeting four weeks in advance of the meeting. A finalized meeting agenda shall be delivered to the technical expert panel two weeks in advance of the meeting. The Contractor shall develop materials, including any background documents and a PowerPoint slide deck for each meeting. Draft meeting materials shall be submitted to the COR five weeks in advance of the meeting, with final meeting materials that incorporate COR comments due two weeks in advance of the meeting. The Contractor, in collaboration with the COR, shall facilitate the technical expert panel meeting. In addition to the federal technical expert panel, the Contractor, in consultation with the COR, shall consult non-federal experts as needed.

If key evaluation or project issues arise, the contractor shall produce brief issue memos on key designs and other study issues as needed over the course of the project.

TASK 2: Evaluation Design and Site Agreements Subtask 2.1 Impact and Implementation Evaluation Design The Contractor shall establish the impact and implementation evaluation designs for the study. The Contractor shall produce an impact evaluation design memo that addresses, at a minimum, the following issues: the research objectives, study settings, the components of the programs to be evaluated, nature and size of target groups for those programs, plan for participant selection and assignment to the intervention and counterfactual group, outcome domains to be measured, plans for impact analyses, and other important aspects of the evaluation.

If randomization to treatment and control groups is not possible, there are multiple options for quasi-experimental designs that could assess the impacts of the demonstration, for example, a difference-in-difference approach using longitudinal data and a comparison group, or a retrospective cohort study using a life events protocol along with multiple follow-up and statistical controls. Use of existing data, such as administrative data if available, is encouraged to create efficiencies in data collection. The contractor shall propose and justify the best approach to providing a rigorous evaluation within budget.

The Contractor shall produce an implementation evaluation design memo. The primary goals of the implementation task are to: (1) document services provided as part of the intervention; (2) identify services provided to participants in the counterfactual group; and (3) describe and document in-depth the intervention design and operations of the participating sites in adequate detail to be useful to future replication of the intervention. In addition to monitoring and tracking the implementation of the treatment services, it will also be critical for the Contractor to closely monitor the services received by participants in the counterfactual group to ensure that there is adequate documentation of the services available to both groups and how the services differ. Aspects of interest for the implementation study include, but are not limited to, the following domains:

· program goals, objectives, and strategies;

· environment in which the program operates;

· program components and services;

· organizational and staff capacities;

· coordination and linkages with other programs or services;

· training and technical assistance received;

· information about services provided in the counterfactual group;

Subtask 2.2: Establish Memoranda of Understanding with Sites The Contractor shall establish and sign a formal MOU with the lead program staff in each site prior to implementation of the evaluation. The Contractor shall conduct visits to the sites selected for the evaluation. These site visits will serve to enable a discussion of the next steps and procedures for the evaluation, including establishing a Memorandum of Understanding (MOU) with the sites. The Contractor shall prepare summaries of the site visits.

The Contractor is responsible for determining: (1) the interests and needs of sites regarding participation in the evaluation; and (2) the roles and responsibilities of the program staff and Contractor. The Contractor shall clearly indicate in a written MOU the roles, responsibilities, and expectations of each party.

The MOU shall include, but not be limited to, the following topics: the proposed evaluation design and work plan; participant assignment procedures; implementation of the intervention in intervention settings; data needs; compensation for study settings to offset financial burdens associated with participation in the evaluation; data rights; data collection procedures; confidentiality and other ethical concerns; process and expectations for publications and presentations using data; a communication and collaboration plan among the program staff and Contractor; the roles and responsibilities of the program staff and Contractor in the evaluation study. A draft of the MOU shall be submitted to the CORs for review and approval. The final approved MOU shall be signed by the lead Contractor and program staff.

TASK 3: Develop Data Collection Plan The Contractor shall develop an overall data collection plan that, at a minimum, describes the expected data to be collected for both the implementation and impact evaluation, data sources, participant assignment procedures, sample recruitment, consent procedures, data collection procedures, participant incentives and time frames for the collection of data that are needed to monitor and assess site progress and performance as well as those needed to conduct the evaluation analyses. The data collection plan shall specify the outcome domains, for which subsequent measures will be defined. The data collection plan must also include, but is not limited to: (1) a strategy for monitoring the implementation of the evaluation; and (2) a comprehensive system for tracking participants over time to ensure a low rate of attrition.

The data collection plan shall indicate data sources for the implementation information (e.g., key informant interviews; review of program-related documents; and intervention and service monitoring). The Contractor shall include in the data collection plan a process and measures for assessing the conditions, policies, and practices within and outside of the program that might influence observed program impacts. For any key informant interviews or focus groups, the plan shall include topic guides for discussions with program staff and participants.

The Contractor shall work with sites to provide study settings with a contact person to call with any questions or issues that arise. The Contractor will keep staff in the study settings informed about the status of the intervention, data collection, and analysis activities as a means of ensuring their cooperation. Furthermore, the Contractor may need to provide an evaluation offset to study settings in order to compensate them for the time and costs associated with participation in the evaluation.

The Contractor shall clearly articulate the procedures to be used to ensure confidentiality of responses and informed consent. The Contractor shall secure Institutional Review Board (IRB) approval. The Contractor shall comply with all requirements imposed by Federal statutes concerning the collection and maintenance of data that includes personal identifiers, in order to satisfy the privacy and confidentiality provisions of the Privacy Act, Social Security Act, and the Health Insurance Portability and Accountability Act (HIPAA). The Contractor shall also comply with all confidentiality and security requirements imposed by FVPSA.

TASK 4: Develop and Pretest Data Collection Instruments and Procedures The Contractor shall develop the data collection instruments indicated in the evaluation design memo and conduct a pretest of data collection instruments and procedures. English and non-English language versions of all data collection instruments shall be developed and tested. Pretesting shall follow the guidelines of the Paperwork Reduction Act of 1995. Data collection instruments shall be submitted to the COR for approval prior to testing. Revised instruments incorporating the COR’s feedback shall be tested and the results shall be described in a technical memorandum to HHS. Final instruments shall be submitted to HHS for inclusion in the OMB package described in Task 3.

TASK 5: Paperwork Reduction Act Package for Submission to Office of Management and Budget The Contractor shall prepare a draft notice of data collection for the Federal Register as required by the Paperwork Reduction Act of 1995. The Act requires publication of such notice 60 days prior to submitting a clearance package to the Office of Management and Budget (OMB). The draft notice shall be provided to the COR. The final notice shall incorporate COR comments.

A draft supporting statement and response to comments submitted under the 60 Day Notice shall be submitted to the COR. The Contractor shall provide a complete OMB clearance package, including a revised draft of the OMB supporting statements as set forth in Standard Form, No. 83a, “Instructions for Requesting OMB Approval,” final data collection instruments that reflect the results of pretesting (in English and non-English languages), a revised version of responses to comments on the 60 Day Notice. The Contractor shall respond to any questions posed or changes requested by OMB during the clearance process. The package shall be considered final after OMB approval is received. The Contractor should budget approximately 9 months from publication of the Federal Register Notice for OMB to provide approval for data collection activities to begin.

TASK 6: Recruit and Train Data Collectors The Contractor shall recruit and train data collectors to implement the data collection plan. To accomplish this, the Contractor shall prepare a recruitment and training plan that describes:

· Number and qualifications of data collectors by type of data collection;

· Approach and schedule for training data collectors; and

· Materials for use in data collector training.

The data collector training plan shall be submitted to the COR and COR comments shall be incorporated into the final plan.

TASK 7: Implement Evaluation Design The Contractor shall identify the sampling frame, carry out sample selection, and set up procedures for maintaining the integrity of the design (i.e., ensure that any control group does not contain participants who receive program components being evaluated). The contractor shall report on Task 5 activities as part of monthly progress reporting under Task 1. The contractor shall deliver a memorandum describing analysis to be conducted to assess equivalency of groups in an experimental design utilizing random assignment, if proposed, or other equivalent analyses to document successful sample design implementation if other methods are proposed.

TASK 8: Data Collection Data collection shall take place in accordance with the procedures approved by the CORs and Office of Management and Budget. The contractor shall report on progress in monthly progress reports under Task 1. The offeror is encouraged to propose use of program data, data linkages, or other efficiencies in obtaining demographic data that could reduce evaluation costs.

TASK 9: Data Analysis Data analysis shall be conducted according to the evaluation design memo and incorporate adjustments as agreed to by the Contractor and COR in writing. The contractor’s plan shall describe plans for conducting nonresponse bias analysis, subgroup analysis, procedures for coding qualitative data, and describe other analytic techniques proposed, including how the primary impacts will be assessed and reported. A preliminary results memorandum shall be delivered to the COR prior to drafting documents reporting those findings.

The contractor shall deliver de-identified data files for all quantitative data, along with full documentation of these data files including:

· File structure (data set name, record format, number of records);

· Code book (record layout including variable name, variable format, variable labels, value labels, and missing values);

· Variable definitions;

· Imputation procedures (if used);

· Editing procedures used; and

· Structure of composite variables.

TASK 10: Briefings and Reports The contractor shall produce a final and summary report, and provide additional interim reports, practice briefs, or other reporting as proposed, with a minimum of annual reporting on the progress of the evaluation, any interim findings, and challenges encountered and strategies for overcoming them.

Final Report. Upon completion of data analysis, the Contractor shall submit a report describing the demonstration and its impacts. This report shall include an executive summary, an introduction with a description of the demonstration background, research objectives and evaluation methods, all evaluation results organized by research objectives, and appendices that provide a technical description of the evaluation methods and analysis procedures. The audience of the final report is the research community. The contractor shall submit an outline, two drafts, and a final version of this report. Each revision shall incorporate feedback provided by the COR. The outline shall go beyond presenting section headers and listing points to be made or topics to be covered and instead shall state key messages.

Summary Report. The Contractor shall produce a stand-alone Summary Report which provides a non-technical overview of the study. Its objective is to make the key findings of the study accessible to non-technical audiences, such as senior policy officials. This report is expected to be no more than 8 pages in length. The contractor shall submit an outline, two drafts, and a final version of this report. Each report shall incorporate feedback provided by the COR. The outline shall go beyond presenting section headers and listing points to be made or topics to be covered and instead shall state key messages.

Final Briefing. The Contractor shall also provide a final briefing for HHS. Draft and final versions of briefing materials shall be submitted for HHS review and approval.

TASK 11: Dissemination The contractor shall execute dissemination activities as proposed after draft and final versions of any slides or other documents to be used are approved by the COR.

OPTIONAL CLINS

Optional CLIN 1: Supplemental Data Collection The offeror shall propose an optional task for supplemental data collection activities, such as additional follow-up to provide another data collection point among study participants.

Optional CLIN 2:

The offeror may propose optional tasks, such as, interim deliverables, innovative dissemination activities, and/or other work that would enhance the usefulness of the evaluation findings for policy and practice but are not possible within the funding available for the base contract.

These optional tasks shall be exercised at the sole discretion of the Government, and may be exercised at any time during the base period of performance.