J.28 Result of application of IMC.pdf

USAID/HONDURAS

PROPONTE MÁS

REPORT UPDATE*: RESULTS OF THE APPLICATION OF

THE INSTRUMENTO DE MEDICION DEL

COMPORTAMIENTO-IMC AT THE NATIONAL LEVEL

May 20, 2020 This publication was produced for review by the United States Agency for International Development and prepared by Creative Associates International INC.

*Updated report: It includes updated analyses after addressing previous errors in the data.

USAID|HONDURAS

PROPONTE MÁS

REPORT UPDATE: RESULTS OF THE

APPLICATION OF THE IMC AT THE

NATIONAL LEVEL

November 2020

SUBMITTED BY:

Creative Associates International

5301 Wisconsin Ave., NW Suite 700

Washington, DC 20015

November 1, 2020

TABLE OF CONTENTS

EXECUTIVE SUMMARY 4

INTRODUCTION 1

DEVELOPMENT AND VALIDATION OF THE IMC 2

NATIONAL SAMPLE 2

I. 3

II. 3

III. 5

Community entry strategy 5

Households were selected systematically and continuously after the first household. The team supervisor defined based on eligibility (households with youths between the ages of 8 to 17 and the presence of a responsible adult to authorize administration) and security criteria. 5

Obtaining consent 5

Georeferencing of the house 5

Control and verification mechanisms during and after administration 5

IV. 6

ANALYSIS OF IMC RESULTS 1

I. 1

II. 2

Socioeconomic and Demographic Characteristics 2

Migration 2

Risky Behaviors 4

Drivers of Risk and the Intention to Migrate 7

Approach 1- Regression Results 9

Approach 2- Structural Equation Model 12

Approach 3-Matching 15

CONCLUSIONS AND RECOMMENDATIONS 17

ANNEX A- METHODOLOGICAL APPROACH 19

EXECUTIVE SUMMARY

Proponte Mas conducted a nationally representative sample of 4,009 youths between 8 and 17 years old to assess their levels of risk using the Risk Behavior Measurement Instrument (IMC by its Spanish acronym). The Proponte Mas team calculated the cutpoints for the full sample of the national IMC and the results show that 57.4% of youth are at a primary level of risk, 41.1% are at a secondary level of risk, and 1.4% are at a tertiary level of risk.

Figure 1. Percentage of Respondents by Risk Factor Differentiation

Source: National IMC Survey

We further disaggregate the descriptive results by gender, age group, race, place of birth (urban/rural), and whether the youth is currently attending school. The results are presented in Table 1 below and show that:

● Males are at a higher risk than females, with 42.3% of respondents being at a secondary risk compared to 39.6% of females.

● Youth 15 to 18 years old and “mestizo” youth are at a higher risk than their peers with 58.1% and 44%, respectively.

● Youth who are out of school are at a higher risk than youth in school, with 65% at a secondary risk than 37% of those currently attending school.

● Youth from urban environments are at a higher risk level, with 43.2% of respondents in urban areas than 33.5% of youth in rural areas.

We conducted additional analyses to examine which characteristics drive youths’ risky behavior and whether being at risk correlates with a youth’s intention to migrate. To do so, we used a series of regression analyses controlling for youth’s social and demographic characteristics that may be associated with a higher probability of being at risk or intending to migrate.

The results for the regression analyses where the outcome of interest is risky behavior suggest, on average, that:

● Males are three percent more likely to be at risk than females.

● Older youths are 14 and 11 percent more likely to be at risk, respectively.

● Youth who self-identify as “Mestizos” are three and five percent more likely to be at risk than their indigenous/afro-descendant peers.

● Youth born1 in urban areas are nine and seven percent more likely to be at risk than their peers born in rural areas.

1 It is likely that this question also captures the place the respondent currently lives.

● Youths who have higher educational attainment appear more likely to want to migrate than their peers with lower educational attainment2.

● Youths whose parents are still alive are seven percent less likely to be at risk than their peers whose parents (one or both) are not alive.

● Finally, the violence measure is highly significant and positively correlated with a higher probability of being at risk. The coefficients are larger in the column with the fixed-effects model. While the marginal effects might seem small, the results indicate that local unobserved factors matter in the impact of violence on the youth’s probability of being at risk.

To estimate the effects on the intention to migrate, we used three methodological approaches.

The first approach employed a probit and ordered probit models. The second approach used a Structural Equation Model (SEM) to measure the direct and indirect effects of explanatory variables on the intention to migrate. Finally, the third approach employed a range of matching estimators to create two groups of youth (at risk and not at risk) similar in observable characteristics. Thus, any differences in the intention to migrate could be attributed to differences in youth’s risk levels.

The results of the first approach indicate the following:

● Those youth being at risk appear more likely to migrate, all else equal. The coefficients are positive, strong, and statistically significant. On average, the results suggest that youth at risk is between ten and 11 percent more likely to intend to migrate.

● Being mestizo does not increase the probability of wanting to migrate compared to other self-identify ethnic groups. The coefficient is positive but not significant at conventional levels.

● Similarly, education, place of birth (urban/rural), having alive parents does not appear to correlate with a higher probability of intending to migrate, all else equal.

● Violence is associated with a higher probability of a youth intending to migrate.

However, this result is only statistically significant in the ordered probit model and the probit model with fixed effects. The magnitude of the coefficient suggests that controlling for other factors, an increase of one homicide per 100,000 inhabitants increases the probability of intending to migrate by 0.003%.

The results of the second approach indicate the following:

In terms of the direct effects of explanatory variables on being at risk:

● Being a male, older, being mestizo, having higher educational attainment, being from an urban area, and higher levels of violence in the municipality are positive and significantly correlated with being at risk

● Conversely, if the respondent’s parents are alive and being from a municipality with migration levels is negative and significantly correlated with the probability of being at risk. In other words, it reduces the likelihood of being at risk.

In terms of the direct effects of explanatory variables on the intention to migrate:

2 This variable measures the respondent’s current year of schooling.

● Being at risk and being in the older youth cohort (15-17) is positively associated with the intention to migrate.

● Being from a municipality with higher migration levels is positive and significantly correlated with a higher intention to migrate.

We did not find any statistically significant indirect effects of the explanatory variables on the intention to migrate through the variable “being at risk,” except for the variables age group and living in a high migration municipality, which was statistically significant at conventional levels (p <.05).

The results for the third methodological approach indicate that:

● There is a positive and statistically significant effect of youth being at risk and the intention to migrate. The coefficients’ size is similar to those in the regression models, ranging from ten to 12 percent.

INTRODUCTION

Proponte Más is a program funded by the United States Agency for International Development (USAID). It is part of initiatives focusing on secondary violence prevention in the Northern Triangle of Central America within the 2015 - 2019 Honduras Country Development Cooperation Strategy, aiming to increase citizen security in urban populations with high crime rates.

Proponte Más’s main objective was to implement a family-based prevention approach that offers new solutions to reduce youths’ risk of joining gangs/maras and becoming gang-related violence victims. Furthermore, the work extended to strengthening the Juvenile Justice System and the empowerment of community structures whose activities contribute to violence prevention.

Between 2016 and 2019, Proponte Más implemented various actions under an evidence-based intervention approach through its five results. One of the project’s intervention core elements focused on using the risk identification tool known as Behavior Measurement Instrument (IMC by its Spanish acronym), which allows identifying youth between the ages of 8 and 17 at the highest risk of exhibiting violent behavior or engaging in criminal activities. The youth who score as “eligible” for secondary prevention services are then referred to a family counseling service provided by Proponte Más.

Proponte Mas’s strategy included calibrating the IMC for its effective utilization by USAID-funded activities in five targeted municipalities (Tegucigalpa, San Pedro Sula, Choloma, Tela, and La Ceiba) for a youth population between 8 and 17 years old. The IMC is a tool that measures the risk in adolescents of being associated with violent or problematic behaviors, 38 factors (11 protective factors and 27 risk factors) within four domains (i.e., community, school, family, peer/individual). In December 2019, the project received a three-month extension to be able to expand the scope of the project and thereby collect a national sample of the IMC. The administering of 4,009 instruments yielded data at the national level through two representative samples. One sample was at the national level, and the other encompassed 12 municipalities with the highest risk of irregular migration per data obtained from the National Migration Observatory and USAID.

It is worth emphasizing the following to the reader:

● The analyses and the findings presented here are considered preliminary. While the cutpoints calculated by the Proponte Mas team were intended for the full sample, given the existing diversity in Honduras, cutpoints for youth in urban vs. rural environments, ethnicity, and other factors, the authors recommend this additional data should be considered to obtain a more accurate understanding of risk levels for youth in Honduras.

● Although the report presents results using methodologies that are often applied in impact evaluations with observable data, these results are not and cannot be interpreted as causality (X causes Y). The reader should interpret them as associations or correlations of an explanatory variable X on Y.

● More research is needed to understand the magnitude and effects of socioeconomic, demographic, and family characteristics on youths’ risky behaviors in Honduras.

This report describes the process to administer the IMC to these two samples, provides details on the methodology, the administration process in some 62 municipalities, the preliminary results from this process, as well as recommendations for using these data in the future.

DEVELOPMENT AND VALIDATION OF THE IMC

The IMC derives from the Youth Services Eligibility Tool (YSET). It emerged from the Gang Risk of Entry Factors (GREF) assessment tool, which was the “generic version” of the YSET that was originally developed by Karen Hennigan and her colleagues at the University of Southern California (for more information, see Hennigan et al., 2015 and Hennigan et al., 2014) through the establishment of a secondary gang prevention program to address high-risk youth in Los Angeles, California (Katz, Cheon, & Zheng, 2019, p. 3).

The assessment tool’s goal was to identify youth who were at high risk for gang joining using nine risk factors.

The YSET was transferred to Honduras in 2013 through a USAID pilot program implemented in five communities in Tegucigalpa, an experience that helped to adapt the tool’s language. In 2016, the USAID Proponte Más project implemented the intervention model on a large scale and conducted the process to adapt, calibrate, and contextualize the tool, which led to the creation of the IMC. The IMC built upon YSET elements and various existing study frameworks dealing with risk factors, problem behavior, and protective factors in developing countries.

Under this framework, the tool was adapted, and some 5,000 instruments were administered to youth between the ages of 8 and 17 in the most violent communities in five municipalities in Honduras. We calibrated the IMC tool using these data through a process that took approximately 12 months.

The Honduran IMC contains various questions about youth, including basic demographics (e.g., age and gender), school attendance and perception, attitudes about police, horizontal family dynamics, vertical family dynamics, parental warmth and support, and victimization. In particular, the IMC includes 173 items that measure 38 factors (11 protective factors and 27 risk factors) within four domains (i.e., community, school, family, peer/individual) (Katz, Cheon, & Zheng, 2019, pg. 5).

Expanding the IMC use through a nationwide sample and an oversample of municipalities with high irregular migration rates responds to the need to continue to increase opportunities that allow calibrating the tool for a future national context. Until now, the IMC has been used in urban contexts highly vulnerable to violence. However, it is unknown whether the survey can be useful in identifying risky behavior in other Honduran contexts, such as rural areas and municipalities with little presence of gang and other forms of crime and violence.

NATIONAL SAMPLE

To obtain the two samples, PM was to subcontract the administration of 4,009 Risk Behavior Measurement Instrument (IMC) interviews in low-income communities across the country under the following scheme:

We obtained two representative samples. One sample was at the national level (49 municipalities) and the other sample comprised 12 municipalities with characteristics that put them at higher risk for irregular migration. The final sample consisted of 4,009 individuals between the ages of 8 and 17. The sample in the 12 municipalities with a higher probability of irregular migration consisted of 1,929 individuals. In comparison, the representative sample (rural and urban stratum) for the rest of the country (the remaining 49 municipalities) consisted of 2,080 individuals. Communities addressed by the municipalities’ project where PM carried out the intervention were excluded to avoid interviewing the same respondents.

I. Selection of the Data Collection Company In January 2020, Proponte Más launched terms of reference in search of a company to carry out the IMC administration process in the 61 selected municipalities. The project received approximately ten proposals from various companies. The technical review process included an assessment of 1) field knowledge; 2) administration methodology; 3) coordination plans in the field; 4) work schedule; and 5) the structure of the personnel who would be administering the instrument.

The company that offered the best competitive advantages for this process was CONFIE, as it proposed a 19-day administration process. This strategic territorial distribution strategy allowed it to meet its objectives under a continuous supervision scheme and start the process right away. This made it a suitable organization to administer the survey instrument given the project’s timeframe for data collection.

The first coordination meeting with CONFIE was held to review the instrument and enter information into https://www.kobotoolbox.org, a free, open-source platform that CONFIE would use to conduct the interviews via tablets. Subsequently, we provided a two-day training workshop to the data collection teams to introduce the tool and explain the use of informed consent and interview protocols. After the workshop, a pilot trial was carried out in Tegucigalpa to test the tool and identify areas for improvement before starting the official survey in the field, which began two days after the pilot. CONFIE deployed 46 interviewers divided into eight field teams, with one supervisor per team.

An additional element of this application was the inclusion of a set of migration questions.

The project organized meetings with the National Autonomous University of Honduras (UNAH) experts through coordination with the University’s Scientific Investigation Directorate (DICU). This process carried out in November, and December 2019 allowed the inclusion of a set of questions derived from the Theory of Planned Behavior to construct this new section included in the IMC.

II. The IMC Tool

The IMC uses 38 measurement scales, one for each risk or protective factor.

Community Risk Factors Transitions and Mobility Low Neighborhood Attachment Community Disorganization Laws and Norms Favorable to Drug Use Perceived Availability of Drugs https://www.kobotoolbox.org/

Community Protective Factors Opportunities for Prosocial Involvement Rewards for Prosocial Involvement School Risk Factors Academic Failure Low Commitment to School School Protective Factors Opportunities for Prosocial Involvement Rewards for Prosocial Involvement Family Risk Factors Family History of Antisocial Behavior Parental Attitudes Favorable Toward Drug Use Poor Family Management Family Conflict Weak Parental Supervision Family Gang Influence Family Protective Factors Attachment Opportunities for Prosocial Involvement Rewards for Prosocial Involvement Peer/Individual Risk Factors Rebelliousness Rewards for Antisocial Involvement Favorable Attitudes Toward Drug Use Favorable Attitudes Toward Antisocial Behavior Perceived Risks of Drug Use Friends’ Use of Drugs Interaction with Antisocial Peer Intentions to Use Antisocial Tendencies Critical Life Events Impulsive Risk-Taking Neutralization of Guilt Negative Peer Influence Peer Delinquency Peer/Individual Protective Factors Belief in the Moral Order Rewards for Prosocial Involvement Interaction with Prosocial Peers Social Skills

Each scale comprises several elements that youth are asked to respond to during an introductory interview. Most of the scales consist of questions with five answer options, ordered from 1 to 5. The answers to items on a given scale are weighted by converting the responses to a scale from 0 to 100 using the Percent of Maximum Possible formula to obtain the respondent’s score. We calculated each by determining the average of the items of all the scales. Scores will always be a number between 0 and 100.

The score is compared with a scale’s risk threshold to determine whether the youth is at risk for that factor or scale. Since the IMC was calibrated for two populations: in and out-of-school youth, each scale’s thresholds are different for each population.

Finally, youth are deemed eligible or ineligible for a family-centered intervention based on the number of risk factors/scales in which the youth scored above the risk threshold.

After determining the risk factors, these are aggregated to determine the youth’s eligibility.

Annex A includes information on the methodological approach and the municipalities where the IMC survey was conducted.

III. Administration guidelines of the IMC The following operational guidelines were used to administer the IMC:

Community entry strategy The field officer and each team supervisor identified and coordinated with authorities and staff from social programs in targeted communities to explain the reason for the visit and the consultancy’s goal. This was backed by an accreditation letter. This activity made it easier for the teams to identify and analyze the security situation, identify the areas within a community where it was possible to work, the most suitable times to conduct this action, and take measures in communities with citizen security issues.

Household selection

Households were selected systematically and continuously after the first household. The team supervisor defined based on eligibility (households with youths between the ages of 8 to 17 and the presence of a responsible adult to authorize administration) and security criteria.

Obtaining consent Once a household with an eligible youth was found, the interviewer explained the purpose of administering the IMC tool, read and explained the informed consent, had the parent or guardian sign the consent form, and explained the purpose of the interview youth.

Georeferencing of the house Once the team supervisor had reviewed the consent forms and checked that they had been signed by both the parent or guardian and the youth and that these included the youth’s date of birth, the supervisor noted the dwelling’s GPS coordinates and assigned an interviewer code and a respondent code.

Control and verification mechanisms during and after administration Survey administration included control and verification mechanisms to ensure the quality of the information. The team supervisor and the field officer were the people responsible for implementing these verification and control mechanisms during and after the IMC tool was administered, which included:

a) Assigning codes to interviewed youths for greater control of survey administration.

b) Household selection consisted of verifying whether the selected household had eligible youth between the ages of 8 and 17 and that the proper consent was obtained.

c) Observation while the IMC was being administered to ensure appropriate privacy was allowed by parents or guardians.

d) Cross-checking georeferencing reports against the information registered in the KoBoToolbox platform.

https://www.kobotoolbox.org/

e) Verification of georeferencing using Qgis and Google Earth software to ensure compliance with the route and zoning plan.

f) Verification of consents against georeferencing reports and classification by municipality.

IV. Administration process of the IMC Below are the issues that arose during administration and the corrective action taken:

Issue Actions

Entry was denied to some neighborhoods due to citizen insecurity issues being experienced in the country.

Coordination with leaders in some communities, including church leaders, women’s groups, etc., or changed to another neighborhood nearby.

Eligible youths were not at home during the visit because they were in school

The team came to the school to speak with school authorities to explain the reason for the visit and showed them the consent forms signed by the youths’ parents or guardians, after which the youths’ consent was obtained to administer the IMC.

Pre-identified areas had changed from a residential to a commercial zoning status, as was the case in Comayagua, for example.

Changed to another neighborhood nearby

Improved financial situation of the population in some neighborhoods (high-income social stratum)

Changed to areas with the initially defined characteristics.

Other comments:

● The staff’s presentation to a youth’s parent or guardian to obtain their consent is extremely important, making them aware of which program it is and who is funding the research. Mentioning that the study was for the USAID Proponte Más project made it easier to get parents and guardians to sign the consent form.

● Future administration of the IMC in the Gracias a Dios area, or anywhere with a strong ethnic presence, will require having staff with extensive knowledge of the language. One of the issues in this Area was that certain communities did not understand Spanish very well.

● Coordination with key actors is a determining factor to apply the IMC since it makes it much easier to move around, especially in urban areas with high vulnerability to violence.

ANALYSIS OF IMC RESULTS

I. Levels of Risk The manual produced by Arizona State’s University showed that the cutpoint to identify risk levels in the Proponte Mas’s sample was as follows:

Primary level of risk: Youth presenting fewer than a total score of 14.5 in the number of risk factors Secondary level of risk: Youth who present 14.5 or more risk factors/scales. For secondary prevention eligibility levels, the total number of risk factors present will determine the specific behaviors that youth are at risk of developing. Predictable behaviors include:

● violence

● property crime

● gang involvement

● drug/alcohol use

● drug sales

● weapon carrying

● truancy (not applicable to the out-of-school population).

Tertiary level of risk: Youth presenting 14.5 or more risk factors/scales, who score above the risk threshold, and answer yes to any of the following three questions:

• In the past six months, have you been a member of a gang?

• Is your group of friends in a gang?

• Are you currently in a crew, clique, or associated with a gang?

Thus, a combination of 14.5 or more risk factors and answering yes to any of the questions above would place youth at the tertiary eligibility level. Proponte Mas used the sample of youth in school and out of school of the full IMC sample3 to estimate the optimal cutpoint. We conducted analyses using the following steps:

● Recalibrate the tool to identify new cutpoints consistent with the national reality of urban and rural diversity, including municipalities with the highest migration rates.

● We conducted an exploratory and confirmatory factor analysis to examine whether the different risk scales fit well with the sub-sample of youth in school and those out of school.

● Once we calibrated the different scales in the tool, we applied ROC4 (Receiver Operating Characteristic) curve analysis to measure specificity and sensitivity to identify the optimal cutpoints.

3 This includes the national sample and the sample of municipalities that have higher propensity for migration.

4 The ROC curve provides a comprehensive performance profile of a diagnostic test (Green and Swets, 1996). The ROC methodology is derived from the signal detection theory, where it is used to determine if an electronic receiver is able to distinguish between the correct signal and noise (Fan et al., 2006). We applied this methodology to assess the effectiveness of the IMC tool in distinguishing between at-risk and non-at-risk youth for various problem behaviors

II. Results

This section of the report presents a summary of the IMC results for the full sample

Socioeconomic and Demographic Characteristics In terms of gender, 57.5% of the respondents were males compared to 42.5 % of females.

Most of the respondents were between 8 and 14 years old (82%) compared to 18% between 15 and 17 years old. Most respondents were born in urban areas (81.5%) than those born in rural areas (18.4%). About 75% of the respondents self-identify as “mestizos” than 26% who are indigenous or afro descendants.

Migration The IMC asked a battery of questions about the intention and attitude towards migration and questions related to having family members living overseas. Because the set of questions asked to respondents are Likert scales, we first examine the distribution of responses in each scale and then a measure that captures a dichotomic response5 on whether the respondent has intentions to migrate, which we use as our outcome variable in our regression analyses. Figure 2 illustrates the distribution of responses to two key questions related to youths’ thoughts about migration. The questions are as follows:

● I feel like going to another country to work and earn money

● I am planning to migrate out of Honduras

These questions relate to the youth’s intentions to migrate. The first question captures the wishes the respondent has to migrate somewhere else to find a better life. The second question captures the actual intentions of the youth to migrate6.

The results show that about 45% of youth respondents completely agree or agree that they would like to migrate compared to 24% of youth respondents who completely agree or disagree about their intention to migrate. Conversely, the percentage of youth who disagree or completely disagree with the questions capturing their wish to migrate and their intention to migrate is 45% and 65%, respectively.

5 This measure is calculated by collapsing the Likert scales at the median.

6 It is worth noting that whether the youth migrates will depend on a wide and complex range of factors.

Figure 2. Likert Scale responses to questions on wanting and planning to migrate

Source: National IMC Survey.

To further examine this result, we disaggregate the question by age group (14 and younger and 15 to 18 years old) and illustrate the results in Figure 3. The results suggest that 32.2% of youth 15 to 18 years old completely agree or agree on the question about their intention to migrate compared to 21.4% of youths eight to 14 years old. Similarly, the question capturing a youth respondents’ wish to migrate indicates that a higher percentage of older youth completely agree with the statement about migrating than 49.7% of younger youth. It is worth noting that differences in responses by age group are statistically significant at conventional levels (p<.05).

Figure 3. Likert Scale responses to questions on wanting and planning to migrate, by Age group.

Risky Behaviors As noted above, the IMC’s overall objective is to identify youth at risk of engaging in risky and delinquent behavior and/or at risk of joining a gang. During the Proponte Mas project implementation, through a subcontract with Arizona State University, the team estimated the cutpoints to identify who is at risk of risky and violent behavior and who isn’t in Honduras. However, it is important to note that we calculated these cutpoints using a previous version of the IMC and a non-representative sample of youth from five municipalities identified by USAID as most at risk of crime and violence per contract requirements. These municipalities included Choloma, La Ceiba, San Pedro Sula, Tegucigalpa, and Tela. These cities are primarily urban centers in Honduras. Therefore, the estimated cutpoints may not be suitable in a nationally representative sample that includes more diversity in the sample, such as urban/rural environments and indigenous and afro-descendent populations7.

Proponte Mas calculated the cutpoint for the national IMC’s full sample using the same approach as ASU’s scoring manual shared with Honduran institutions. We calculated the cut point for each of the 38 risk and protective factors by problem behavior (see the section Levels of Risk). We used the lowest score to estimate the cut point between being at primary risk, secondary or tertiary risk.

After estimating the new cutpoints, figure 4 illustrates the percentage of youth at risk in Honduras. The results indicate that 57.4% of youth are at a primary level of risk, 41.1%

7 It is worth noting that Proponte Mas worked with Garifuna communities, but these populations were not the main objective of the program.

are at a secondary level of risk, and 1.4% are at a tertiary level of risk. These results, for example, contrast with the findings of Proponte Mas’s cutpoints. Proponte Mas’s sample’s estimated cutpoints are different from those found in the national IMC sample. In the Proponte Mas’s sample, the percentage of youth at a secondary level of risk was between 20% and 25% compared to 41.1% in the national sample.

Figure 4. Percentage of Respondents by Risk Factor Differentiation

It is important to note to the reader, however, that the association of the various dimensions of risk and protective factors with each problem behavior differs from that of Proponte Mas’s sample analyses. For example, compared to Proponte Mas, the ROC analyses suggest that some of the risk and protective factor dimensions do not predict problem behaviors as expected8. Thus, these dimensions were dropped from the final count of cumulative risk factors to identify risk categories.

The analysis undertaken in the Proponte Mas’s sample included a cutpoint analysis for the sample of children and youth in school and those out of school. We conducted the same estimations in addition to the estimation of cutpoints for the full sample. This underscores the importance of continuing to evaluate this tool and identify cutpoints to identify risk levels using different domains. For example, a researcher could conduct the analyses by comparing youth living in urban/rural areas, male/female, or their intentions to migrate.

Figure 5 presents the ROC curve analysis for the risk cut points’ mean score for all problem behaviors. The resulting cut point is 9.5 risk factors, and the Area Under the

8 Note that the analysis undertaken for the Proponte Más sample also had some dimensions that the ROC analysis was not able to discriminate a true positive from a false positive.

Curve (AUC) is 0.65, which indicates a somewhat low precision9 (Fischer, Bachman, and Jaeschke, 2003), but in line with previous research on this topic10.

To classify youths by risk level, a cutpoint equal or greater than 9.5 places a youth at a secondary risk level (see Figure 2 above); and a cutpoint equal or greater than 9.5 plus a positive answer to the question on gang membership places a youth at a tertiary risk level.

Figure 5. ROC Curve Analysis

We then disaggregated the cutpoints by gender, age group, race, place of birth (urban/rural), and whether the youth is currently attending school. The results are presented in Table 1 and show that:

● Males are at a higher risk than females, with 42.3% of respondents being at a secondary risk compared to 39.6% of females.

● Youth in the older age cohort (15-18) and “mestizo” youth are at a higher risk than peers, with 58.1% and 44%, respectively.

● As expected, youth out of school are at a higher risk than youth in school, with 65% at a secondary risk than 37% of those currently attending school.

● Finally, youth living in urban environments are at a higher level of risk, with 43.2% of respondents in urban areas than 33.5% of youth in rural areas.

9 The ASU’s manual indicates the following: “The greater the AUC, the better the test. Theoretically, the maximum value for the AUC, which is 1.0, indicates a perfect diagnostic test (i.e., 100% of sensitivity and 100% of specificity). An AUC value of 0.5 indicates no discriminative value (i.e., 50% of sensitivity and 50% of specificity). In general, ROC curves with an AUC greater than 90% indicates high accuracy, while 70% - 90% shows moderate accuracy, 50%-69% demonstrates low accuracy (Fischer, Bachman, and Jaeschke, 2003). Given this, optimal cut-points were selected as the point maximizing AUC value over all possible cut-point values”.

10 As a point of reference Singh et al. (2011) reported among multiple prior studies the median area under the curve for the following nine risk assessment tools: SVR-20=.78; SORAG=.75; VRAG=.74;

SAVRY=.71; HCR-20=.70; SARA=.7-; Static-99=.70; LSI-R=.67; and PCL-R=.66) (ASU, 2019).

Table 1: Disaggregation by Risk Levels

Primary Secondary Tertiary

Gender

Male 56.3 (1,292) 42.27 (970) 1.44 (33)

Female 58.93 (1,010) 39.61 (679) 1.46 (25)

Age Group

8-14 62.73 (1,961) 36.34 (1,136) 0.93 (29)

15-18 38.62 (341) 58.1 (513) 3.28 (29)

Race/Ethnicity

Mestizo 53.77 (749) 44.01 (613) 2.23 (31)

Indigenous/Afro-Descendants 59.37 (1,553) 39.6 (1,036) 1.03 (27)

Currently Studying

Yes 61.11 (2,153) 37.84 (1,333) 1.05 (37)

No 30.66 (149) 65.02 (316) 4.32 (21)

Place of birth: Rural-Urban

Urban 55.41 (1,750) 43.19 (1,364) 1.39 (44)

Rural 64.82 (36) 33.54 (18) 1.63 (1)

Source: National IMC Survey.

Notes: For this report, we calculate the percentages by disaggregating the full sample by gender, age group, race, urban/rural, and school attendance. Ideally, and depending on the interest of the program implementer, a cutpoint should be estimated for one or various groups. As indicated above, Proponte Mas estimated the cutpoints for the full sample – used in this report—and for those youth in and out of school. Differences between gender, age group, race/ethnicity, currently attending school, and place of birth are statistically significant (p<.05).

Drivers of Risk and the Intention to Migrate In addition to collecting data on risk and protective factors at the national level, another key objective of the IMC was to collect information on youths’ intentions, attitudes, and subjective norms about migration.

This section presents the econometric analyses on the factors that drive youths’ risky behaviors and their intention to migrate irregularly from Honduras. The empirical approach for the analyses is as follows:

For the analysis of the drivers of risk, we modeled it by creating a dichotomous variable of who is at risk and who isn’t. We then ran a probit model to measure the effect of social and demographic variables on being at risk. We also ran a fixed-effects model to control for unobservable city characteristics that could affect the outcome.

Finally, we included two additional variables in the models. The first variable is a measure of violence—municipal homicides and the homicide rate per 100,000 inhabitants—to examine its effect on both the risk of violence and the probability for a youth’s intention to migrate.

The second variable is a dummy that captures whether the respondent lives in a municipality with a higher probability of migration. The reader might recall that the IMC data collection exercise included a representative sub-sample of youths in municipalities that, according to USAID, had higher migration levels.

For the migration analysis, we conducted our empirical analysis using three different approaches. The first approach used a Likert scale type question on the intention to migrate as the dependent variable in two different models—Ordered probit and probit.

The reason for this approach is twofold. An ordered probit model seems more appropriate because the “distance” between the responses, which range from “Completely Agree” to “Completely Disagree,” may not be the same. Therefore, it is important to ascertain the effect that those explanatory variables have on each of the item responses.

A probit model requires using a dichotomous measure as the dependent variable—values range from 0 to 1— and would capture changes in the dependent variable from 0 to 1, which might be easier to interpret for the reader.

For this report, we ran the models discussed above using the responses to the following question as our dependent variable in the regression analyses because it more closely captures the youth’s actual intention to migrate:

-Estoy planeando irme de Honduras – I am planning to leave Honduras11.

The second model specification employs a dichotomous (0/1) measure using the Likert-type question “I’m planning to leave Honduras” after collapsing the variable at the median12.

The second methodological approach employed a structural equation model (SEM) to estimate the direct and indirect effects of explanatory variables and the variable capturing risk on the intention to migrate. SEM models allow the researcher to examine

The third approach used a range of matching estimators to create a sample of youth who intend to migrate and youth who do not13 that are similar based on a range of observable characteristics such as age, gender, race, educational status, and place of birth. By creating two similar samples, any differences in the “intention to migrate” could be attributed to a youth being at risk14.

For this report, we only present stylized facts and do not provide an in-depth discussion (nor speculate) on the reasons that may be driving these results. To do this would require additional analyses.

11 The Likert scale values were reversed to reflect a range that goes from negative to positive instead of positive to negative as it was coded in the original dataset.

12 This is a common procedure to generate dichotomous variables out of Likert scale measures.

13 The variable was used 14 The reader should note that this is not an impact evaluation, and the results should not be interpreted as such.

Approach 1- Regression Results Table 2 presents the regression results of the drivers of risk. The estimates suggest that all else, equal:

● Males are three percent more likely to be at risk than females.

● Older youths are 14 and 11 percent more likely to be at risk, respectively.

● Youths who self-identify as “Mestizos” are three and five percent more likely to be at risk than their indigenous/afro-descendant peers.

● Youths born15 in urban areas are nine and seven percent more likely to be at risk than their peers born in rural areas.

● Youths who have higher educational attainment appear more likely to want to migrate compared to their peers with lower educational attainment. This variable captures the respondent’s current year of schooling.

● Youths whose parents are still alive are seven percent less likely to be at risk than their peers whose parents (one or both) are not alive.

● Finally, the violence measure is highly significant and positively correlated with a higher probability of being at risk. The magnitude of the results is stronger in the column with the fixed-effects model. While the marginal effects might seem small, the results indicate that local unobserved factors matter in the effects of violence on the probability of youth being at risk.

Table 2. Regression Results-Being at risk of various problem behaviors

(1) (2) VARIABLES Marginal effects-Drivers of risk Marginal effects-Drivers of

Risk-FE

Gender = 1, Male 0.0348** 0.0305* (0.0172) (0.0181) Age Group= 15-18 0.137*** 0.117*** (0.0323) (0.0337) Race/Ethnicity= “Mestizo” 0.0342* 0.0533*** (0.0179) (0.0196) Year of School = 2 0.105 0.106 (0.108) (0.0944) Year of School = 3 0.118 0.138 (0.101) (0.0860) Year of School = 4 0.108 0.107 (0.100) (0.0865) Year of School = 5 0.149 0.156* (0.100) (0.0848) Year of School = 6 0.130 0.139

(0.0998) (0.0872) Year of School = 7 0.191* 0.205** (0.0999) (0.0855) Year of School = 8 0.224** 0.246*** (0.101) (0.0863)

15 It is likely that this question also captures the place the respondent currently lives.

Year of School = 9 0.257** 0.269*** (0.101) (0.0891) Year of School = 10 0.249** 0.269*** (0.103) (0.0889) Year of School = 11 0.277** 0.283*** (0.108) (0.0982) Year of School = 12 0.279** 0.318*** (0.111) (0.0978) Urban/Rural 0.0964*** 0.0711** (0.0247) (0.0278) Parents Alive -0.0722*** -0.0719*** (0.0138) (0.0151) Homicide Rate per 100,000 inhabitants

0.000444*** 0.0999**

(6.42e-05) (0.0465) Living in a High Migration Municipality

-0.0747** -2.878**

(0.0293) (1.464) Observations 3,476 3,476

Standard errors in parentheses; *** p<0.01, ** p<0.05, * p<0.1 Notes: Probit model with fixed effects (2) to control for unobserved heterogeneity at the municipality level.

The estimated coefficients capture the discrete change of binary variable from 0 to 1.

Source: Analyses based on the National IMC Survey.

Table 3 presents the results of the regression analyses on the intention to migrate measures. For ease of interpretation, we discuss only the results of the probit model16 (columns 2 and 4—see the paragraph on the next page with an explanation). The results indicate the following:

● Those youth being at risk appear more likely to migrate, all else equal. The coefficients are positive, strong, and statistically significant. On average, the results suggest that a youth who is at risk is between ten and 11 percent more likely to want to migrate.

● Being mestizo does not increase the probability of wanting to migrate compared to other self-identify ethnic groups. The coefficient is positive but not significant at conventional levels.

● Similarly, education, place of birth (urban/rural), having alive parents does not appear to correlate with a higher probability of intending to migrate, all else equal.

● Violence is associated with a higher probability of a youth intending to migrate, but this result is only statistically significant in the ordered probit model and in the probit model with fixed effects. The magnitude of the coefficient suggests that controlling for other factors, an increase of one homicide per 100,000 inhabitants increases the probability of intending to migrate by 0.003%.

16 The ordered probit model results are available upon request from the author.

Table 3. Regression Results-Intention to Migrate

(1) (2) (3) (4) VARIABLES Intention to

Migrate- Ordered Probit

Intention to Migrate-

Probit

Intention to Migrate-

Ordered Probit-

FE

Intention to Migrate- Probit-FE

Being at Risk 0.241*** 0.111*** 0.227*** 0.106*** (0.0378) (0.0167) (0.0403) (0.0174) Gender = 1, Male 0.0432 0.00461 0.0580 0.00580 (0.0370) (0.0181) (0.0387) (0.0189) Age Group= 15-18 0.127* 0.0820** 0.103 0.0803**

(0.0750) (0.0343) (0.0754) (0.0341) Race/Ethnicity= “Mestizo”

-0.0920** 0.00828 -0.0789* 0.00380

(0.0396) (0.0178) (0.0451) (0.0197) Year of School = 2 0.113 0.0356 0.186 0.0621

(0.203) (0.123) (0.210) (0.125) Year of School = 3 -0.0792 -0.120 -0.0196 -0.113

(0.189) (0.115) (0.200) (0.118) Year of School = 4 -0.123 -0.0991 -0.0541 -0.0856

(0.191) (0.116) (0.200) (0.118) Year of School = 5 -0.130 -0.112 -0.0593 -0.0985

(0.189) (0.115) (0.199) (0.118) Year of School = 6 0.00722 -0.0672 0.0783 -0.0519

(0.188) (0.115) (0.198) (0.118) Year of School = 7 -0.0676 -0.0911 0.00368 -0.0772

(0.188) (0.115) (0.199) (0.118) Year of School = 8 0.000571 -0.0360 0.0771 -0.0236

(0.195) (0.117) (0.205) (0.120) Year of School = 9 -0.126 -0.101 -0.0401 -0.0790

(0.202) (0.119) (0.213) (0.122) Year of School = 10 0.0103 -0.0650 0.0904 -0.0500

(0.193) (0.121) (0.204) (0.124) Year of School = 11 -0.0588 -0.103 0.0243 -0.0971 (0.211) (0.121) (0.221) (0.125) Year of School = 12 -0.000168 -0.0969 0.0538 -0.104

(0.224) (0.128) (0.231) (0.129) Urban/Rural 0.0517 0.0347 0.00507 0.00456 (0.0559) (0.0259) (0.0618) (0.0283) Parents Alive -0.00354 -0.00552 -0.0149 -0.00944

(0.0316) (0.0140) (0.0322) (0.0145) Homicide Rate per 100,000 inhabitants

0.00124** 0.000367 0.000842 0.00375**

(0.000623) (0.000304) (0.00421) (0.00170) Living in a High Migration Municipality

0.0281 0.0372* -0.107 -0.208

(0.0421) (0.0211) (0.250) (0.137) /cut1 -0.708*** -0.571** (0.225) (0.261) /cut2 0.699*** 0.858*** (0.228) (0.263) /cut3 1.042*** 1.204*** (0.228) (0.263) /cut4 1.946*** 2.116*** (0.229) (0.264)

Observations 3,476 3,476 3,476 3,476

Standard errors in parentheses; *** p<0.01, ** p<0.05, * p<0.1 Notes: Probit model with fixed effects (2) to control for unobserved heterogeneity at the municipality level.

The estimated coefficients capture the discrete change of binary variable from 0 to 1.

Columns 3 and 4 in Table 3 employ a fixed-effects model (we discuss only the results in column 4)—the reader may recall that this type of model controls for unobservable city characteristics that could affect the outcome. The results remain very similar, and the effects of being at risk on the intention to migrate are slightly adjusted downward.

It is worth noting the effect of the homicide rate on the intention to migrate. The coefficient is positive and statistically significant (albeit the effect is a bit weak). To interpret this result, an increase in one homicide per 100,000 inhabitants would increase the probability of a youth’s intention to migrate by 0.3%.

Similar to the results presented in Table 2, the influence of local unobserved factors matters in the impact of violence on a youth’s intention to migrate.

Approach 2- Structural Equation Model

This report also examined the direct and indirect effects of demographic characteristics and violence on risk and migration using a structural equation model approach (SEM).

SEM models are useful because they allow us to identify the direct effects of socioeconomic and demographic variables on the intention to migrate and being at risk and, indirectly, on migration through being at risk. A mediated effect indicates that and the correlation of an explanatory variable with the outcome is transmitted through a third variable, which is a youth being at risk.

To illustrate this with an example, we know that age directly affects the intention to migrate since younger people tend to want to migrate more than older people. At the same time, there is also a direct correlation between age and being at risk as young people may be more at risk, given a set of socioeconomic and demographic characteristics. Therefore, under this example, age would be directly correlated with the intention to migrate and indirectly correlated with the intention to migrate through the being at risk measure. As a result, the variable measuring a youth’s risk levels could increase (or decrease) the effect of age on a youth’s intention to migrate.

Figure 6 illustrates the SEM model we used for the analyses. The outcome of interest is the intention to migrate, and our mediating/moderating variable captures the youth’s level of risk.

Figure 6. Structural Equation Model.

Table 4 presents the results of the analyses and are similar to those in Table 3. We summarize them in the following lines:

In terms of the direct effects of explanatory variables on being at risk:

● Being a male, older, being mestizo, having higher educational attainment, being from an urban area, and higher levels of violence in the municipality are positive and significantly correlated with being at risk.

● Conversely, if the respondent’s parents are alive and being from a municipality with higher migration levels is negative and significantly correlated with the probability of being at risk. In other words, it reduces the likelihood of being at risk.

In terms of the direct effects of explanatory variables on the intention to migrate:

● Being at risk and being in the older youth cohort (15-17) is positively associated with the intent to migrate.

● Being from a municipality with higher migration levels is positive and significantly correlated with a higher intention to migrate outside Honduras.

In general, the analyses did not find any statistically significant effects of the explanatory variables on the intention to migrate through being at risk, except for the variables age group and living in a high migration municipality.