J.1.8_EF Performance Eval Exec Summary DRAFT.pdf

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 1

MONITORING AND EVALUATION SUPPORT FOR

COLLABORATIVE LEARNING AND ADAPTING

(MESCLA) ACTIVITY

PERFORMANCE EVALUATION: HONDURAS WORKFORCE

DEVELOPMENT ACTIVITY

MONITORING AND EVALUATION SUPPORT FOR

COLLABORATIVE LEARNING AND ADAPTING

(MESCLA) ACTIVITY

SUBMITTED TO:

USAID/Honduras

Prepared by:

Dexis Consulting Group

DISCLAIMER: This is a DRAFT of the Executive Summary, which includes major findings, along with Section 2: Evaluation Purpose and Evaluation Questions of the recent performance evaluation of USAID/Honduras’ current Workforce Development Activity, Empleando Futuros. It is important to note, however, that the Youth Employment for Migration Prevention RFP is considerably different in goals, particularly in terms of scale and methodologies for private sector engagement and contributions.

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 2

1 EXECUTIVE SUMMARY

1.1 OVERVIEW

USAID/Honduras contracted Banyan Global (Banyan) to implement the Empleando Futuros (EF) Workforce Development (WFD) Activity in Honduras. This Activity aimed to increase citizen security for vulnerable populations in urban, high-crime areas in Honduras by supporting workforce development to increase income-generating opportunities for youth who are the most at risk of being perpetrators of violence. The Activity strengthened comprehensive workforce readiness services, including for job linkage and self-employment, to benefit at-risk youth, including those who qualify for secondary and tertiary violence prevention services. The WFD Activity in Honduras contributed to the USAID/Honduras Country Development Cooperation Strategy Development Objective 1 (DO1), Sub-Intermediate Result 1.1.2, “Quality services that protect against violence increased.” All participants were from communities with medium to high levels of violence in Tegucigalpa, San Pedro Sula (SPS), Tela, Choloma, La Lima, Villanueva and La Ceiba. The WFD Activity focused primarily on young males, given their prevalence in national violence statistics, both as perpetrators and victims of violence. At the time of enrollment, target participants were out of school, with no less than a sixth-grade education and no more than a high school degree; and unemployed, underemployed, or with inconsistent or low-skill employment (a population known as “NiNis” in Spanish).

The WFD Activity originally aimed to train 7,500 participants, of whom 60% were to be male and 30% assessed as at secondary-risk-level or above. The WFD Activity sought to insert 3,750 of those trained into the workforce. Due to funding cuts in 2019, the program revised training targets to 4,600 and insertion targets to 2,500 during the period of evaluation (see Country Context below). To provide lessons learned to WFD to make any adjustment necessary to reach their goal, USAID/Honduras requested the Monitoring and Evaluation Support for Collaborative Learning and Adapting (MESCLA) Activity to conduct a performance evaluation (PE) of the WFD Activity. A first evaluation of performance was conducted on the WFD Activity performance through June 30, 2018. This second and final PE focuses on those participants enrolled in the WFD Activity from April 2019 to December 2019.

Together with WFD Activity implementing partners, MESCLA collected participant data using surveys at program intake and upon completion. MESCLA also followed up with program deserters and conducted key informant interviews (KIIs) with participants, implementing partners, and employers. The results were examined to determine areas for adaptation and learning for future WFD programs. According to program reports, by the end of December 2019, 3,421 participants had completed the WFD program, and 1,137 participants were employed or had returned to formal education at some point during the 12 months following completion of training.

1.2 COUNTRY CONTEXT

Performance during the evaluation period was impacted by the March 30, 2019 announcement of funding cuts to the region in response to high rates of illegal migration to the United States.

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 3

USAID/Honduras requested scaling back training and activities under EF results. Scheduled funding was ultimately withheld during the remainder of FY19 which resulted in changes to the training targets.

After a dramatic increase leading up to May 2019, migration attempts decreased dramatically, falling by more than 90% to less than 2,000 per month by May 2020, compared to more than 40,000 in May 2019

(CBP/DHS 2013-2020).

Employment during the same period for those between 19-24 and 25-29 was 87.8 and 91.7 percent respectively, while official unemployment was only 12.2 and 8.3 percent (INE, 2019). Meanwhile, those who were underemployed or working for less than the minimum wage was 66 percent for both groups.

This demonstrates the fact that, despite low official unemployment, there is a great number of youths who work but do not earn enough to sustain themselves or their families.1

Finally, the Covid-19 pandemic began as data collection for this performance evaluation was beginning.

Changes were made to capture activities and employment before the national lockdown on March 15, 2020. Forecasts about the impact of the pandemic on the labor market were negative,2 however the exact medium- and long-term impact of the pandemic and lockdown on responses cannot be known with the data collected. (Participants were asked about employment before the lockdown in an attempt to minimize the impact on responses. See further discussion in the Limitations section).

1.3 KEY FINDINGS

Some findings referenced here and throughout the report use dropouts (did not complete 80 percent of course, n=54), basic training completers (n=58), and technical training completers (basic + additional technical training) (n=466) for comparison purposes only when results are statistically significant. When comparisons are made, those comparisons are only suggestive of difference/no difference between groups given that there was no way to control for potential selection bias. Pre-post results without a comparison group should also be interpreted with the normal caution for performance evaluations, as it is not possible to attribute changes to the WFD Activity alone since no impact evaluation was undertaken.

1.3.1 EMPLOYMENT AND EMPLOYABILITY

● The improvement in completion rates indicates that the WFD Activity modifications improved the retention of participants across all socio-demographic characteristics (sex, age, or education

1 In 2010 a “transitory” decree was issued for a special temporary employment program No. 230-2010 (Gazette No. 32, 358 of 11/05/2010) establishes temporary employment for specific work or services. The duration of the contracts can be up to 36 months and are renewable. At the beginning of 2014, Congress issued the Hourly Employment Law making temporary employment modalities included in the 2010 program permanent. According to a 2018 UNAH report on the employment situation in Honduras, the hourly employment law - instead of fulfilling its purpose of reducing unemployment -caused an increase in underemployment in the country. The visible underemployment rate increased from 4.6% in 2009 to 14.2% in 2018.

Visible underemployment by definition includes people who worked less than 36 hours in the reference week and at the time of the interview expressed a desire to have been able to work longer and is used as a proxy for temporary employment.

2 https://forbescentroamerica.com/2020/05/19/cohep-alerta-de-posible-desaparicion-de-400000-empleos-en-honduras-por-covid-19/ https://forbescentroamerica.com/2020/05/19/cohep-alerta-de-posible-desaparicion-de-400000-empleos-en-honduras-por-covid-19/ https://forbescentroamerica.com/2020/05/19/cohep-alerta-de-posible-desaparicion-de-400000-empleos-en-honduras-por-covid-19/

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 4 level) since the mid-term PE. In the mid-term PE completion rates differed by gender (women completed at higher rate than men), age (lower completion for 16-17-year-olds, highest for 18- 19-year-olds), and education level.

● Seven in ten completers who were unemployed when they started the program were employed or had returned to school in the six-month follow up.

● Nearly half of those who completed technical training and engaged in some kind of economic activity after the WFD Activity indicate that their employment was better (in terms of pay, stability, hours, or location) than before. This improvement is especially notable given the high rates of underemployment in Honduras. This improvement is statistically significant when compared to less than a quarter of those who did not complete technical training reporting better employment after the WFD Activity.

● About one quarter of those who were in informal employment or self-employed when the program started had moved to a formal position in the six-month follow up, and an additional one in ten returned to school (this was true of dropouts as well, meaning this move cannot be attributed solely to the WFD Activity).

● Eight in 10 dropouts who were unemployed when they started the program were employed or had returned to school in the six-month follow up. One in every three dropouts left the program because they found a job or to return to school.

● Despite inconsistent direct job placement support from EF reported by participants, participants noted that the training they received from EF improved their job search and interview skills, improved their entrepreneurial abilities, and provided them with a recognizable certification that aided in their own job search.

● Among program completers, women are significantly less likely than men to be employed or to have returned to school at the six-month follow up (76.4 percent versus 63.4 percent, p<0.05).

However, at the national level, women are also less likely to be employed or in school than men:

89.7 percent of men versus 56.8 percent for women.)3.

● Individuals in the age group 26 to 30 had 53 percent lower odds of being employed at the six month follow up than those in the age group 16-20, whereas individuals with lower resilience levels were 24 percent less likely to intend to migrate than those with higher resilience levels.

● Women, those with less education at baseline, those who completed only basic training, and younger participants, reported more improvement in employability skills than men, those with more education at baseline, those who completed technical training, and older participants. This is not surprising given that the employability skills measured are basic labor competencies.

● Nearly all (13 of 17) employers interviewed expressed satisfaction with EF employees and cited better attitudes/soft skills among EF graduates as reason for hiring or desiring to hire from the program in the future.

● Service providers cite poor attitudes and the vulnerable situation of youth as major challenge to program completion.

3 These numbers, estimated using INE’s 2018 household survey, consider those who are not working for pay or looking for employment as not working.

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 5

● Soft skills and positive attitudes are key for success in the program and in securing and maintaining employment (while employers mentioned appropriate technical skills as an area for improvement).

● Service providers are using or plan to use the lessons learned, materials (especially life skills), and relationships (with other service providers and employers) in other programs.

● The most common area for improvement mentioned participants was clarity in the services offered. Participants felt cheated because, when enrolling, they understood the WFD Activity would help them find employment or a job interview, and that had not happened in all cases.

1.3.2 VIOLENCE

● Participants with secondary and tertiary risk of engaging in violent behaviors at the baseline had a higher probability of dropping out of the WFD Activity than those with primary risk.

● Nearly six in ten participants at secondary or tertiary risk levels at baseline decreased their risk between baseline and the six-month follow up.

● Technical training completers showed the most consistent and statistically significant improvements in protective factors for violence involvement (namely emotional regulation and resilience).

1.3.3 MIGRATION

● Participants with secondary education and those in the primary risk category were less likely (statistically significant) to have attempted to migrate in the year prior to enrolling in EF than those with no secondary education and those in the secondary risk category.

● Mirroring national migration trends, migration intentions decreased by nearly two thirds between baseline and six-month follow up, regardless of program completion.

● Migration intentions decreased for dropouts and those who completed basic training, but not for those who completed technical training, which were already substantially lower at baseline than overall trends (all statistically significant).

● The reason for having migration intentions remained stable with lack of employment as the most common reason for having migration intentions at both baseline and six-month follow up.

● Migration intentions increase with past migration attempts and decrease with perception of a good future in Honduras.

● About three out of four participants agree that there is a good future in Honduras.

● Perception of a good future in Honduras increases among those who completed basic or technical training compared to dropouts.

● Perception of a good future in Honduras decreases by age group.

● Similar to those with migration intentions, nearly 9 in 10 of those who do not see a good future in Honduras cited the economic situation as the primary reason for that negative perception.

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 6

2 EVALUATION PURPOSE & EVALUATION QUESTIONS

2.1 EVALUATION PURPOSE

The Empleando Futuros WFD Activity in Honduras began in FY2016 under Development Objective 1 Sub- Intermediate Result 1.1.2, “quality services that protect against violence increased”. USAID/Honduras contracted Bayan Global to implement the WFD Activity from FY2016-20214. The Activity is part of the Central America Regional Security Initiative (CARSI). As such, it aims to support workforce development that will increase income-generating opportunities for youth who are the most at risk of being perpetrators of violence in urban, high-crime areas in Honduras. The WFD Activity will strengthen comprehensive workforce readiness services, including job linkage and self-employment, to benefit at-risk youth, including those who qualify for secondary and tertiary violence prevention services.

The WFD Activity goal in Honduras is to “increase employment and protective factors for at-risk youth living in the targeted USAID/Honduras DO1 high crime municipalities of Tegucigalpa, San Pedro Sula, Choloma, La Ceiba, and Tela.”

The WFD Activity seeks to achieve the following three primary results:

● Result 1: Access to high quality, comprehensive workforce development services for at-risk youth increased, with an emphasis on youth who qualify for secondary prevention services.

● Result 2: INFOP’s institutional capacity to deliver high-quality, market-driven services improved.

● Result 3: Access to workforce-related services, including income-generating activities, increased for youth who have been in conflict with the law, including former gang members.

At-risk youth eligible to participate in the WFD Activity were defined by the following criteria: (1) living in selected high-crime communities in the DO1 target municipalities of Tegucigalpa, San Pedro Sula, Choloma, La Ceiba, Lime, Villanueva and Tela; (2) being out of school, with at least a sixth-grade education and no more than a high school degree; and, (3) being unemployed, under-employed, or with inconsistent or low-skill employment. The criteria also specify that at least 30% of the participants will be at-risk youth in need of secondary prevention services.

4 In response to cuts to United States foreign assistance to Honduras, in July 2019 Banyan Global submitted a contingency plan to USAID outlining adjustments in activities (reductions in scale and scope) and modified projections due to the change in available funds. Formally accepted by USAID in August 2019, that plan indicates that project activities will continue through November 2020.

Theory of Change

At-risk youth who are provided with high-quality, comprehensive, and market-driven vocational training services and assistance (life skills, basic competencies, psycho-social support, technical skills and job placement support) will both increase their employment opportunities and reduce their risk factors;

therefore, increasing their household incomes and reducing their incidence for being victims or perpetrators of violence and crime.

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 7

The scope of this PE covers Result 1 exclusively. The PE will serve to support USAID’s decision-making process and identify delivery strategies to ensure outcome effectiveness in subsequent activities.

It should be noted that a pilot impact evaluation was conducted in 2017 but was terminated due to challenges with consistency in implementation and high dropout rates in the first full year of the WFD Activity. While improvements have been made in both areas, the PE is expected to provide further input for future adaptation. Absent a control group, however, the PE will not be able to answer the impact evaluation questions originally presented by USAID.

2.2 EVALUATION QUESTIONS

According to the MESCLA EF Technical Proposal (9/2017), the PE process was to answer two learning questions:

● Q1: To what extent are quality WFD services increased, and to what extent do quality WFD services protect against violence?

● Q2: To what extent do WFD actions produce a workforce with relevant skills to support country development?

The mid-term PE (2018/19) focused on the service delivery and violence portions of Q1 given that the status of program implementation did not allow for a review of employment/employability. The current PE will therefore focus on Q1 sub-questions dealing with employment/ employability (i.e. special focus on employment/employability in Q1.2-1.4) and Q2.

The scope of this PE is limited to the following sub-questions:

● Q1.1: To what extent has the WFD Activity reached the targets of Result 1? Why have these targets been or not been reached?

● Q1.2: How does the risk of violence and employability change after program completion and six months after finishing the program?

● Q1.3: What are the perceptions of beneficiaries about the people and services that help them succeed?

● Q1.4: How do stakeholders perceive the quality of the WFD Activity deliverables?

● Q2.1: To what extent is the WFD Activity delivering job placement services to participants?

● Q2.2: What are the perceptions among service providers about the services of the WFD Activity?

● Q2.3: What are the perceptions among employers about the services of the WFD Activity?