FAM-25-039 Attachment M VDSS Regional Post Adoption Services Evaluation _Final Report_December 2024.pdf

VDSS Regional Post-Adoption Services Evaluation

Final Report

Berenice Rushovich, Sharon Vandivere, Kenya Downing

1 | P a g e

Table of Contents Executive Summary

Background and purpose

Key activities

Reports, training, and presentations

Key findings and recommendations

Suggestions for ongoing evaluation

Background and Purpose

Initial consultation: Phase 1

Evaluation: Phase 2

Synopsis of Activities

Key activities and findings

Reports, training, and presentations

Recommendations

Suggestions for Ongoing Evaluation

Acknowledgments

Appendices

Appendix A: VDSS Post-Adoption Consultation Summation 2020

Appendix B: VDSS Evaluation Report 2023

Appendix C: VDSS Post Adoption Logic Model

Appendix D: VDSS Reporting Toolkit #1

Appendix E: VDSS Reporting Toolkit #2

Appendix F: Regional Consortium Reports Review

Appendix G: Post Adoption Case Management: Common Elements and Models

Appendix H: Adoption Services: Pre-Finalization through Post-Adoption

Appendix I: VDSS Staff Training: Front line workers

Appendix J: VDSS Staff Training: Supervisors

Appendix K: VDSS Post Adopt Consultation Project

Appendix L: VDSS PAC Evaluation Report Presentation

December 2024

2 | P a g e

Executive Summary

Background and purpose Beginning in 2018, the Virginia Department of Social Services (VDSS) launched a new effort to improve post-adoption support and services in the Commonwealth to strengthen adoptive families and promote their stability. VDSS contracted with Child Trends to embark on a two-phase research process to inform their work. This document provides a short overview of both phases of the work, a brief description of the findings from each phase, a summary of the products developed from the work, and recommendations for ongoing evaluation and continuous quality improvement (CQI).

In the first phase, VDSS aimed to understand how best to ensure that families, regardless of where they live, have access to high quality, complementary, and integrated array of services. Based on the research findings in the first phase, Child Trends made recommendations for a comprehensive, integrated model of adoption support. This model of support included plans for the establishment of five Regional Post-Adoption Consortiums (RPAC) with a lead provider agency in each. In April 2022, VDSS contracted with Child Trends to evaluate the RPAC model in the second phase of work, which aimed to assess whether the changes to post-adoption services may have influenced adoptive families’ strength and stability. This report focuses on the work conducted in phase 2.

Key activities We began phase 2 with a review of literature and administrative data, and primary data collection on changes since implementation of the RPAC model. Below, we list the key activities conducted in phase 2.

• Reviewed current literature to identify the root causes of adoption instability.

• Created a logic model to frame the work.

• Conducted a surveys and interviews to understand staff, parent and young adult perspectives on:

o Service use and gaps.

o Staff adoption competence and needs for additional support

• Developed two toolkits to support staff practice:

o Toolkit #1: Post-adoption best practices toolkit.

o Toolkit #2: Enhancing adoption competent practice.

• Reviewed RPAC processes and models to understand:

o Ways to enhance reporting and data collection.

o Standard case management through adoption of a tested model.

o Pre-finalization to post-adoption process.

Reports, training, and presentations We summarized and presented the findings to VDSS, Local Department of Social Services (LDSS) staff, RPAC leads, and Permanency Advisory Committee members in reports, trainings, and webinars. Below is a list of reports, training, and presentations.

• Reports on findings.

• Training for staff on evaluation findings and adoption competence.

• Presentations o RPAC meeting attendance.

o Adoption Month Presentation.

o Presentations to the PAC.

3 | P a g e

• Evaluation findings.

• Pre-finalization to post-adoption review.

Key findings and recommendations Key findings and recommendations based on our phase 2 work are summarized below.

• The RPAC is a helpful model for providing services and worth sustaining.

• The steps in the handoff process for families from pre-finalization to post-adoption services could be clearer for staff and families.

• Families could benefit from a consistent, manualized, evidence-informed case management model, such as Success Coach.

Suggestions for ongoing evaluation We appreciated the opportunity to provide consultation on the formation of the RPAC comprehensive, integrated model of adoption support, and evaluate the initial implementation of the model. To support the ongoing success of the model, we offer recommendations for an implementation and outcome evaluation of Success Coach as well as a repeat evaluation of the RPAC Success Coach model. Research questions and evaluation activities are listed below.

RPAC with Success Coach Implementation questions

• Are RPAC lead agencies able to implement Success Coach with fidelity to the model?

o Are there differences in implementation among the RPAC leads, by agency and/or by region?

o What accounts for these differences?

• What facilitates the successful implementation of Success Coach?

• What barriers did agencies encounter during the implementation of Success Coach?

o How, and with what level of success, were these barriers addressed?

Outcome questions

• How is receipt of Success Coach associated with (1) subsequent child maltreatment reports and (2) re-entry into foster care/agency custody?

o How does the association compare to subsequent child maltreatment reports and re-entry into care/custody in the years prior to implementation of Success Coach?

• Do children and families who receive Success Coach services experience improvements in well-being and other select measures of functioning after services compared to before service?

• Do program outcomes differ by family characteristics (e.g. differing ages of children at time of adoption; child with pre-existing emotional or behavioral challenges; differing placement types—kin compared to non-kin adoption)?

• Do program outcomes differ across VDSS regions or between rural and other regions?

Repeat evaluation of RPAC with Success Coach model for CQI and outcome purposes

• Conduct ongoing performance management.

o Focus on rapid cycle testing, such as a Plan, Do, Study, Act process o Continue to ensure the voices of youth who were adopted, and adoptive parents are included.

o Continue to use data gathered—including changes in families’ needs, use of services, satisfaction with services, and gaps in services—to inform practice and make course corrections.

• Conduct an ongoing implementation study.

https://www.ahrq.gov/health-literacy/improve/precautions/tool2b.html

4 | P a g e o Continue to measure fidelity to the RPAC with Success Coach model, including:

▪ Changes related to service availability and delivery.

▪ Changes in collaborative efforts and joint service provision.

• Track outcomes regionally, as well as by rural versus urban (or non-rural).

o Tracking data can help uncover differences in RPAC implementation that may be related to local conditions, providers, or other factors.

o With knowledge of regional differences, VDSS and the regional consortia can formulate and implement changes to address challenges and improve service delivery.

5 | P a g e

Background and Purpose Beginning in 2018, the Virginia Department of Social Services (VDSS) launched a new effort to improve post-adoption support and services in the Commonwealth to further strengthen adoptive families and promote their stability. VDSS contracted with Child Trends, a private non-profit research and evaluation company, to embark on a two-phase research process to inform their work. Although this report provides a short overview of both phases of the work, including a brief description of the findings and recommendations, the bulk of the report is dedicated to documenting the work carried out in phase 2.

Further details on work carried out in phase 1 can be found in VDSS Post Adoption Consultation Summation 2020, Appendix A.

Initial consultation: Phase 1 To inform VDSS’ efforts, phase 1 included a needs assessment and evaluation of the services currently offered to families post adoption. The needs assessment and evaluation Child Trends conducted strategically focused on the underserved geographic areas within the Commonwealth. Child Trends also examined how collaborations between organizations can strengthen the array of services, such as collaborations among client advocacy groups that represent foster, kinship, and adoptive families, and public and private services providers that prepare children and parents for adoption.

The needs assessment began with a review of the current post-adoption services and support across Virginia and literature regarding evidence-based and evidence-informed post-adoption programs. The review of services and literature informed the development of a comprehensive, integrated model of adoption support, that is based on an expansive service array, to meet the needs of adoptive families across a continuum, including suggested competencies for service providers and adoptive parents. Child Trends then reviewed VDSS administrative data to provide a comprehensive picture of the families VDSS serves.

We gathered data from post-adoption service providers and adoptive parents through interviews, focus groups, and surveys to evaluate the current array of services and identify any strengths, weaknesses, and gaps. We used this information to refine and enhance our recommended service array and tailor it to the specific Virginia context.

Findings resulting from Phase 1 work At the time of our analysis, Virginia had approximately 7,000 children adopted from foster care, an additional 7,000 children adopted internationally, and an unknown number—likely the largest of the three groups—adopted privately from within the United States. Many adoptive parents in Virginia had support in their community and family. They also tended to have adequate financial resources, and many were well educated (over 80 percent had post-secondary education).

Although not all families need post-adoption services to succeed, many families face challenges, and all should be prepared to recognize when to seek help and how to obtain it when needed. Some challenges that adoptive families face are related to biological and experiential factors pertaining to children prior to adoption, and some are normative challenges related to adjusting to adoption. Nearly all children who have experienced foster care have histories of maltreatment.1 For this reason, it is likely that children adopted

1 Spinazzola, J., Habib, M., Knoverek, A., Arvidson, J., Nisenbaum, J., Wentworth, R., Hodgdon, H., Pond, A., & Kisiel, C. (2013). The heart of the matter: Complex trauma in child welfare, CW360o: Trauma-Informed Child Welfare Practice. Winter: 8–9, 37

6 | P a g e from foster care are more likely than the general population to have mental health challenges, meaning some may require post-adoption services.2

Our findings from the key informant interviews, focus groups, and surveys were generally consistent with the literature on challenges facing adoptive families as well as the literature on the support and services needed to help stabilize and maintain the viability of these adoptive homes. Most providers and adoptive parents agreed about which services are lacking and which services they want and need to be successful and for their children to thrive. Below, we describe recommendations for robust post-adoption services and support in Virginia, as well as the key findings that support those recommendations.

Recommendations resulting from Phase 1 work Based on the research findings, Child Trends made recommendations for a Comprehensive, Integrated Model of Adoption Support. The principles supporting the model include:

• Support must represent a full continuum of services to meet the array of families’ needs.

• Services must be available to all families at any time up until their adopted children are no longer minor dependents.

• Service providers must have necessary skills/competencies, including a strong understanding of adoption as well as challenges common among adoptive families.

• Families must be actively engaged over time.

• Services must be available and accessible across geographic regions.

• A “navigator” (i.e., a single point of contact that can assess the family’s need and can connect the family to the appropriate resource) should be available.

• Families must have access to a state-wide website where they can find appropriate, available, and accessible services.

• Qualified provider agencies must be able to implement the integrated model of adoption support.

To implement an Integrated Model of Post-Adoption Support, Child Trends recommended that VDSS specify related requirements in the VDSS post-adoption Request for Applications (RFAs), as follows:

• Establish five regional Post Adoption consortiums with a lead provider agency in each. Consortium-wide communication could facilitate cross-region learning and standardization.

• During implementation, track Commonwealth-specific adjustments needed, given that recommendations built on not only Viginia-specific research, but also on the national state of the field and research.

• Leverage agencies that are already familiar with their regions’ needs and resources.

• Use technology to promote access to services.

• Require the five RPAC lead agencies to jointly develop and manage a statewide website.

• Provide pre-finalization preparation for parents and children to help prevent problems from escalating, reducing the need for more costly or intensive services.

• Make education, such as adoption competency training, available to providers on an ongoing basis.

• Provide technical assistance through communications products and strategies targeting public and private child welfare leadership, services provider organizations, adoptive families, and broader public audiences.

• Monitor capacity and performance of all VDSS-funded services providers, including the extent to which providers have the necessary skills and competencies related to adoption.

2 Bramlett, M.D., Radel, L.F., & Blumberg, S.J., (2007). The health and well-being of adopted children, Pediatrics, Volume 119: S54–S60. https://doi.org/10.1542/peds.2006-2089i https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1542%2Fpeds.2006-2089i&data=05%7C02%7Cbrushovich%40childtrends.org%7Ce49bb286501e451f7fd608dd1f7c6949%7C380c6d8fdce34747b5fda656050bfd7f%7C0%7C0%7C638701341390387656%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=IiHPY8IRVUF1L%2FzU%2BP1%2Fh2ofI4k7AfE2%2BnQ3uMu9F58%3D&reserved=0

7 | P a g e

Evaluation: Phase 2 VDSS contracted with Child Trends in April 2022 to assess any changes related to the new way of providing post-adoption services through the RPAC. Child Trends provided evaluation consultation based on the following research goals:

1. Review changes in the root causes of adoption dissolution and disruption since 2019.

2. Understand whether the post-adoption support and services may be influencing changes in adoption cases coming to the attention of the child welfare system (prevention and placement cases), including the ways the RPAC is or is not influencing the rate of disturbed1 adoptions.

• Understand whether the RPAC is associated with changes in services and support received by families and outcomes for these families.

3. Understand what post-adoption services and support families need to prevent disturbed adoptions (this includes prevention and intervention services offered by any entity, including VDSS, LDSS, and private RPAC partners).

4. Understand the extent to which RPAC members have the needed adoption skills and competencies and how that plays a role in the effectiveness of post-adoption services.

5. Recommend ways to enhance and improve post-adoption services, including identifying which post-adoption services should be prioritized for funding.

Details and outcomes of the Phase 2 work are described in the remainder of this report, below.

Synopsis of Activities

Key activities and findings Phase 2 of the work included a review of literature and administrative data, and primary data collection on changes since implementation of the Regional Post Adoption Consortium (RPAC) model. In this section, we summarize key findings from our evaluation and describe activities including development of a logic model to frame the RPAC goals and objectives and toolkits to support practice. For more details about our methods and findings, please see the VDSS Evaluation Report 2023, Appendix B.

Review of root causes of adoption instability Child Trends’ review of Virginia and federal data as well as our review of the recent literature indicated that neither the youth population in out-of-home placement, nor the root causes of adoption instability have changed much since our previous report in 2019. The factors that reduce the risk of adoption instability also remain consistent, emphasizing a need for educational and informational services, clinical services, material services, and peer support from those with lived adoption experience. See Appendix B for more information.

Creation of a logic model to frame the work Child Trends developed a logic model (See Appendix C) to describe the assumptions behind the VDSS post-adoption service model, resources available, post-adoption activities and inputs, outputs and short-term outcomes, and intermediate and long-term outcomes. A logic model is a concise display of how a program achieves its intended goals and can be useful for periodically ensuring the program is still on track to meet the intended goals and for identifying if and where adaptations might be needed.

2 | P a g e

Incorporation of staff, parent, and young adult voices Child Trends administered two surveys:

• One to parents who had adopted a child and live in Virginia regarding their adoption experiences, use of, and satisfaction with adoption-related services pre- and post-RPAC implementation.

o This survey was a repeat of the survey we conducted in the first phase of the study, however both surveys were anonymous, so we do not know if they were the same or different parents who completed each survey.

• One to staff at LDSS offices and RPAC member agencies regarding their level of adoption competence as well as their understanding of the RPAC model and changes to services post-RPAC implementation.

We also interviewed parents and staff about their adoption experiences, including their use of and satisfaction with services pre- and post-RPAC implementation. Below are a few key findings from the surveys and interviews. For more information, please refer to Appendix B.

Use of, and gaps in, services Parents in general reported little change in their use of services or satisfaction with services received pre- to post-RPAC implementation. It is important to remember that the RPAC model began during the COVID-19 pandemic, when many services were still being provided virtually, rather than in person. This may have affected how parents accessed services as well as their satisfaction with services. Although many services were provided virtually during the data collection period, some services were provided in person. In comparison to findings from a survey conducted during the first phase prior to RPAC implementation (2018- 2020), the parents we surveyed in the second phase reported receiving more support post-adoption after RPAC implementation. Parents who completed the second survey reported a lower sense of their efficacy as parents related to understanding their children’s feelings and problems when compared to parents who completed the first survey, although on both surveys parents reported feeling effective as parents in general.

In the staff survey, staff rated which services they thought were the most available, affordable, and accessible for families. The top five responses were peer supports, recreational resources, educational, medical, and concrete services. Staff also rated which services families faced the most barriers to accessing, and the top five were mental health, disability-related, childcare, financial, and respite services.

Staff and parents also commented on the post-adoption service array. Staff and parents valued the RPAC model because it provides a consistent point of contact between staff and families and a centralized location from which to send information to families. Parents valued a consistent case manager who was available and accessible to them, and especially one who had lived adoption experience. They also valued support groups and other family events, among other enhanced post-adoption services. Parents most frequently reported challenges accessing and using crisis support, including community crisis responders and hospital services.

Staff adoption competence and need for additional support Overall, staff rated themselves as possessing a high level of competence in adoption-related values, knowledge, and skills. Staff who had master’s level training and more years of experience reported higher levels of competence. Additionally, staff reported that the biggest barriers to developing, and applying competent practice were a lack of time to devote to developing knowledge and resistance from families to the services they offer.

7 | P a g e

Toolkits to support practice Child Trends translated the evaluation findings into two toolkits to support staff and inform practice improvement. The first toolkit is for RPAC leads and others who need to track service outputs and outcomes, and the second toolkit is for staff and supervisors working with adoptive families.

Toolkit #1: Post-adoption best practices toolkit The Post-Adoption Best Practice toolkit provides tools to evaluate implementation and outcomes and to maintain and/or improve effectiveness of the consortium model. See Appendix D for the toolkit.

Toolkit #2: Enhancing adoption competent practice The Enhancing Adoption Competent Practice toolkit provides information and resources on the current needs of adoptive families; the competencies required of staff—including values, knowledge, and skills to serve families; and the training necessary to enhance adoption competence. See Appendix E for the toolkit.

Review of reporting processes and service models Child Trends reviewed the following processes and models and provided recommendations for enhancements: RPAC reporting and satisfaction materials; the current RPAC case management model; and the process from families pre-finalization to post-adoption. For more details, please see the following reports, which were submitted previously to VDSS: Regional Consortium Reports Review (Appendix F);

Post-Adoption Case Management: Common Elements and Models (Appendix G); and Adoption Services:

Pre-Finalization Through Post-Adoption Process PowerPoint (Appendix H). A summary of the work completed follows.

Ways to enhance reporting and data collection We reviewed the RPAC reporting tools, including the annual and monthly reports and client satisfaction surveys, to understand the utility of the data being collected and what data were missing. We then provided recommendations for how the data collection, reporting, and analysis processes could be enhanced.

Standard case management through adoption of a tested model From our review and synthesis of the literature on case management models, we identified 11 elements commonly mentioned as part of case management as well as five additional elements typically offered in conjunction with case management. We then interviewed the RPAC leads about the elements of case management they offered and barriers they encountered in implementing case management. We then presented three case management models for consideration, including the strengths, weaknesses, replication support and training, and cost of implementation for each model. After VDSS selected the Success Coach model for adoption, we introduced VDSS to the Success Coach developers and attended Success Coach informational meetings with the RPAC leads.

Pre-finalization to post-adoption process Families and staff reported a gap between services available to families pre-adoption finalization and services available post-adoption, when problems may arise. To identify specific weaknesses in services, we reviewed the current process as outlined in VDSS’ February 2022 Child and Family Services Manual: Section F Post Adoption Services, as well as the Adoption Through Collaborative Partnership documents and the RFP # FAM-20-082 detailing RPAC services. We synthesized the materials and summarized the roles and responsibilities of LDSS staff, RPAC staff, and families related to service access and delivery. We gathered input from VDSS and members of the Permanency Advisory Council and used this input to develop recommendations for enhancement to the process.

https://www.dss.virginia.gov/files/division/dfs/ap/intro_page/manuals/02-01-2022/section_4_post-adoption_services.pdf https://www.dss.virginia.gov/files/division/dfs/ap/intro_page/manuals/02-01-2022/section_4_post-adoption_services.pdf

8 | P a g e

Reports, training, and presentations Below, we describe the reports, trainings, and presentations we created to disseminate the findings and promote use of toolkit #1 to different audiences.

Reports We submitted the following reports to VDSS, based on our findings described above:

• Rushovich, B., Muñoz , K.S., Vandivere, S., Fojut, J., Downing, K. & DeCoursey, J. (2023). Virginia Department of Social Services, Division of Family Services: Report on Post-Adoption Consultation Findings.

Rockville, MD: Child Trends. (See Appendix B)

• Vandivere, S & Rushovich, B (2023): Review of Consortium Reporting Tools: Annual and Monthly Reports and Client Satisfaction Surveys. Rockville, MD: Child Trends. (See Appendix F)

• Rushovich, B. (2024): Post-Adoption Case Management: Common Elements and Models Report.

Rockville, MD: Child Trends. (See Appendix G)

Training for staff on evaluation findings and adoption competence We presented two trainings entitled “Enhancing Adoption Competent Practice:” one to frontline LDSS and RPAC staff on August 6, 2024 (Appendix I), and one to LDSS and RPAC supervisors on August 8, 2024 (Appendix J). The training was recorded and posted on the VDSS training website.

Presentations We presented our findings to the Permanency Advisory Council and to attendees of an Adoption Month special meeting. We attended RPAC monthly meetings to present findings and gather feedback. These presentations are described in more detail below.

RPAC meeting attendance Beginning in September 2023, we attended monthly RPAC meetings. During meetings, we took notes, presented findings from our evaluation, and gathered feedback from RPAC leads, which we then incorporated into our reports.

Adoption month presentation On November 15, 2022, we presented a webinar entitled “VDSS Post-Adoption Consultation Project” (see Appendix K) highlighting the work Child Trends has done over the past six years, including a description of the project, findings, recommendations, and opportunities for participation.

Presentations to the Permanency Advisory Council

Evaluation findings

On May 15, 2024, we presented our evaluation findings based on our evaluation report to the Permanency Advisory Council. See Appendix L.

Pre-finalization to post-adoption review

On November 20, 2024, we gave a presentation to the Permanency Advisory Council entitled “Adoption Services: Pre-Finalization Through Post-Adoption Process” (see Appendix H). We gathered input from staff

9 | P a g e via a Miro Board activity and summarized the suggestions in an email to VDSS, which are included in our recommendations below.

Recommendations Based on our work over the past six years, we offer some final recommendations around sustaining the RPAC model, enhancing the handoff from pre-finalization to post-adoption services, and implementing a consistent case management model, such as Success Coach.

The RPAC is a helpful model for providing services and is worth sustaining.

Families value services received through RPAC, so continue to fund the RPAC model to ensure families have access to free services designed to support safe and stable adoptive placements. Families especially value having a consistent case manager, and preferably one who has lived experience with adoption and has the necessary knowledge, skills, and values in adoption practice.

Staff believe they are adoption competent but could benefit from additional training in cultural competence.

Staff who had master’s level training and more years of experience reported higher levels of competence. It is likely true that more training and experience translate to higher levels of competence, although this is based on staff self-report, not verified evidence. Nevertheless, we think it would benefit agencies to hire well-trained staff and create working conditions that encourage them to stay in their positions over the long term.

VDSS values accurate and useful data collection to support implementation of the RPAC model. The existing reporting tools used to collect information from RPACs have some drawbacks, and the following suggestions may enhance data accuracy and usefulness:

• Create checklists for reporting when possible.

• Continue to provide open-ended response fields to provide context to responses.

• Consider using a survey platform to gather the information instead of an Excel workbook.

• Provide a glossary of terms to ensure shared understanding among RPACs.

• Make the monthly and annual reports consistent with each other.

• Ask RPAC leads for their input on the reports.

• Collect data about differences in experiences with services across subgroups of children and families.

• Develop data collection methods to facilitate comparisons across regions to better understand the unique needs of each region.

• Examine the rate at which agencies use RPAC funds.

VDSS aims to enhance consistent provision of services across the Commonwealth’s regions, especially in rural areas. Regular (monthly or quarterly) joint RPAC lead check-ins have proven to be a useful way for regional leads to share their procedures and develop common definitions of services. We recommend continuing these meetings on a regular basis. In addition, to identify regional differences, we recommend gathering input from RPAC staff, through a survey or short interviews, on the types and intensity of services they provide. These differences can then be addressed during the RPAC lead check-ins.

The handoff from pre-finalization to post-adoption services could be smoother.

Families and staff agree that there is a need for improving the handoff of families from staff who provide pre-finalization services to those who offer post-adoption services and support. Families reported they sometimes struggle with knowing how and where to access services post-adoption. PAC members offered valuable suggestions for enhancing the handoff, and we added a few of our own to the list:

10 | P a g e

• Ensure families know about RPAC services prior to finalization and are referred to RPAC services when needed.

o Consistently and actively market to families in ways that engage them.

o Include description of post-adoption services in pamphlets about adoption, and ensure materials include statements to normalize help seeking post-adoption.

o Include more, clear, and concise information on the public post-adoption website.

o Reach out to the community, including schools, so they are aware of RPAC services.

o Include RPAC services in the adoption information sheet during the adoption assistance process.

o Make use of the RPAC video produced for the VDSS adoption worker training.

o Add information about RPAC services to the annual affidavit mailings.

• Train staff in the adoption continuum and families’ service needs from pre-finalization to post-adoption.

o Define all staff roles and responsibilities during training, supervision, and in post-adoption manuals.

o Offer joint trainings on adoption competent practice to LDSS and RPAC staff.

o Encourage regular check-ins and presentations regarding services during staff meetings.

o Develop brief “micro” learnings geared towards increasing knowledge and use of RPAC services.

o Continually review materials and trainings to ensure they reflect the cultures and backgrounds of all families.

o Encourage consistent attendance of LDSS and private agency staff at RPAC meetings where post-adoption services are discussed, so all staff are informed about the available service array.

• Hire more staff to focus solely on adoption and post-adoption services.

Families may benefit from a consistent, manualized, evidence-informed case management model, such as Success Coach.

One of the goals of the RPAC model is to ensure families in all parts of Virginia have access to a standardized array of services, including access to high quality case management. The Success Coach case management model has been shown to support high quality case management and has a strong theoretical framework guiding the work. It has a well-developed implementation manual that includes fidelity measures, the developers can train and support staff through adopting the model and implementing it, and it would allow VDSS the ability to measure outcomes for families across regions.

Suggestions for Ongoing Evaluation

Implementation and outcome evaluation of Success Coach We appreciated the opportunity to provide consultation on the formation of the RPAC comprehensive, integrated model of adoption support, and evaluate the initial implementation of the model. To support the ongoing success of the model, we offer recommendations for an implementation and outcome evaluation of Success Coach as well as a repeat evaluation of the RPAC Success Coach model. Research questions and evaluation activities are listed below.

An implementation evaluation provides information about how and how well agencies adopt and integrate a program model, such as Success Coach, into their practice. If the model is not being implemented correctly, then it is unlikely to result in the desired outcomes. As such, it is important to know if the model is

11 | P a g e implemented as intended and without unnecessary or unsanctioned modifications. Findings from the implementation evaluation could be used to identify and rectify factors that may be preventing full implementation. It is also important to understand the factors that support and challenge implementation for program and continuous quality improvement (CQI) purposes.

An outcome evaluation provides information on changes recipients experience that may be related to the services they receive, such as changes in families’ experiences related to Success Coach case management.

One possible way of measuring outcomes is by using existing data, such as the “disturbed” adoptions report, to compare outcomes for children prior to implementation of Success Coach with those of children whose families receive Success Coach services. Another way to measure outcomes is by conducting pre- and post-service assessments to examine whether families’ functioning improves on a pre-selected range of measures after receiving services. Below are possible questions that could be addressed through an implementation and outcome evaluation.

Implementation questions

• Are RPAC lead agencies able to implement Success Coach with fidelity to the model?

o Are there differences in implementation among the RPAC leads, by agency and/or by region?

o What accounts for these differences?

• What facilitates the successful implementation?

• What barriers did agencies encounter during the implementation of Success Coach?

o How, and with what level of success, were these barriers addressed?

Outcome questions

• How is receipt of Success Coach associated with (1) subsequent child maltreatment reports and (2) re-entry into foster care/agency custody?

o How does the association compare to subsequent child maltreatment reports and re-entry into care/custody in the years prior to implementation of Success Coach?

• Do children and families who receive Success Coach services experience improvements in well-being and other select measures of functioning after services compared to before service?

• Do program outcomes differ by family characteristics (e.g. differing ages of children at time of adoption; child with pre-existing emotional or behavioral challenges; differing placement types—kin compared to non-kin adoption)?

• Do program outcomes differ across VDSS regions or between rural and other regions?

Repeat evaluation of RPAC with Success Coach model for CQI and outcome purposes

• Conduct ongoing performance management.

o Focus on rapid cycle testing, such as a Plan, Do, Study, Act process.

o Continue to ensure the voices of youth who were adopted and adoptive parents are included.

o Continue to use data gathered—including changes in families’ needs, use of services, satisfaction with services, and gaps in services—to inform practice and make course corrections.

• Conduct an ongoing implementation study.

o Continue to measure fidelity to the RPAC with Success Coach model, including:

▪ Changes related to service availability and delivery, and

▪ Changes in collaborative efforts and joint service provision.

• Track outcomes regionally, as well as by rural versus urban (or non-rural).

https://www.ahrq.gov/health-literacy/improve/precautions/tool2b.html

12 | P a g e o Tracking data can help uncover differences in RPAC implementation that may be related to local conditions, providers, or other factors.

o With knowledge of regional differences, VDSS and the regional consortia can formulate and implement changes to address challenges and improve service delivery.

Acknowledgments We would like to acknowledge the input and feedback we received from adoptive parents, young people who were adopted, and LDSS and RPAC staff. The information in this report would not have been possible to gather and verify without their assistance. We also want to thank VDSS for your guidance and for giving us the opportunity to work with you over the last six years to really get to understand the post-adoption services landscape in Virginia. Finally, we would like to thank Jan DeCoursey for her thoughtful and helpful input.

13 | P a g e

Appendices

Appendix A: VDSS Post-Adoption Consultation Summation 2020

Appendix B: VDSS Evaluation Report 2023

Appendix C: VDSS Post Adoption Logic Model

Appendix D: VDSS Reporting Toolkit #1

Appendix E: VDSS Reporting Toolkit #2

Appendix F: Regional Consortium Reports Review

Appendix G: Post Adoption Case Management: Common Elements and Models

Appendix H: Adoption Services: Pre-Finalization through Post-Adoption

Appendix I: VDSS Staff Training: Front line workers

Appendix J: VDSS Staff Training: Supervisors

Appendix K: VDSS Post Adopt Consultation Project

Appendix L: VDSS PAC Evaluation Report Presentation

Suggested Citation Rushovich, B., Vandivere, S., & Downing, K. (2024). VDSS Regional Post-Adoption Services Evaluation: Final Report.

Child Trends