J.26 Longitudinal Analysis of perceptions.pdf

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 1

MONITORING AND EVALUATION SUPPORT FOR COLLABORATIVE

LEARNING AND ADAPTING (MESCLA) ACTIVITY

HONDURAS CONVIVE! PROGRAM:

LONGITUDINAL ANALYSIS OF

PERCEPTIONS OF SAFETY AND SECURITY

May 2018 Contract Number: AID-OAA-I-15-00019– Task Order Number: AID-522-TO-16-00002

This publication was produced for review by the United States Agency for International Development. It was prepared independently by Dexis Consulting Group and the University of Notre Dame Initiative for Global Development and authored by Juan Carlos Guzman and Tom Hare.

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) i

MONITORING AND EVALUATION

SUPPORT FOR COLLABORATIVE LEARNING

AND ADAPTING (MESCLA) ACTIVITY

Honduras Convive! Program:

Longitudinal Analysis of Perceptions of Safety and Security

VERSIONS:

• Draft 1 – February 15, 2018

• Draft 2 – February 27, 2018

• Draft 3 – March 8, 2018

• Draft 4 – April 23, 2018

• Final Draft – May 3, 2018

SUBMITTED TO:

USAID/Honduras

Prepared by:

Dexis Consulting Group

DISCLAIMER:

The authors’ views expressed in this document do not necessarily reflect the views of the United States Agency for International Development or the United States Government.

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 1

TABLE OF CONTENTS

Table of Contents

Acronym List

1 Executive Summary

2 Analysis Purpose & Questions

2.1 Analysis Purpose

2.2 Analysis Question

3 Activity Background

3.1 Context

4 Methods & Limitations

4.1 Data

4.2 Variables

4.3 Estimation approach

4.4 Limitations

5 Results

5.1 Victimization

5.2 Problems in the community

5.2.1 Drugs/ drug trafficking

5.2.2 Violence

5.2.3 Gangs

5.2.4 Robbery

5.3 Perpetrators of violence

5.3.1 Gang Members

5.3.2 Thieves

5.4 Change in Perception of security in the community

5.5 Opinion on National Police performance in the community

6 Key Findings and Conclusions

7 Appendix I: Tables and HLM Results

HLM Econometric Model

9 Appendix II: Questionnaire Review

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 2

Acronym List

GOH Government of Honduras

HLM Hierarchical Linear Model

INL Bureau of International Narcotics and Law Enforcement

MEL Monitoring Evaluation and Learning

MESCLA Monitoring and Evaluarion Support for Collaborative Learning and Adapting

OTI Office of Transition Initiatives

US United States of America

UN United Nations

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 3

1 EXECUTIVE SUMMARY

USAID/Honduras, through the Office of Transition Initiatives’ (OTI) Honduras Convive!

Program (“Convive” or “the Program”), has supported perception surveys of high crime communities for five years (2013-2017). The surveys were conducted by the Program1 both in communities that have benefited from OTI interventions (treatment communities) as well as in communities where the program was going to be implemented in a later year (“control communities” as defined by Convive). The primary objective of these surveys was to better understand community perceptions of safety and security, and the effect USAID and the Government of Honduras (GOH) programming have on these perceptions. Understanding the potential impact of this program is critical to ensure successful replication to disrupt the systems, perceptions, and behaviors that support violence. The Program will end in 2018, and USAID/Honduras seeks to capture lessons learned and highlight achievements of the Program.

Key Findings

• From 2013-2017, homicide rates dropped in all Convive intervention municipalities and rose slightly in the comparison municipality of Choloma.

• Reported victimization decreased in all communities (Convive and comparison) from 2013-2017.

• The perception of frequency of drug use, drug trafficking, violence, and gangs decreased in Convive communities at a faster rate than comparison communities from 2015-2017, though comparison communities were beginning to catch up toward 2017. These perceived improvements took place even though the homicide rate leveled off or increased slightly from 2015-2016 in some communities.

• Perceptions of security improvements were modestly better in Convive communities than in treatment communities from 2013-2017, with the greatest improvement in Convive communities compared to comparison communities in 2015 and 2017. These perceived improvements may have been bolstered by decreased homicides in all included communities (except those in La Ceiba) from 2013-2015.

• Perception of police performance increased slightly in all communities (Convive and comparison) from 2013-2017.

2 ANALYSIS PURPOSE & QUESTIONS

2.1 ANALYSIS PURPOSE

The purpose of the analysis is to:

• Assess, through econometric analysis, the impacts that the Program has had on community perceptions of safety and security by comparing changes over time within and between the intervention and comparison communities for the period 2012–2017.

• Analyze the changes over time in perception and self-reporting of victimization and reporting of crimes within and between the intervention and non-intervention communities, determining whether the changes are statistically significant.

1 The ex-post longitudinal analysis in this report was requested after all waves of data collection had occurred under the direction of the Program.

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 4

• Assess the perceptions on violence and insecurity, police performance, and the main problems affecting the community between intervention and non-intervention communities for the period of analysis, and other insecurity/security indicators over time in the treatment and control communities.

• Assess the extent of change in the treatment and control communities in community perception of security over the past five years as relates to other available security and insecurity indicators (i.e. homicides) over time.

Other considerations

• The project was not designed with an impact evaluation in mind. An impact analysis requires data from a control group that is collected during the life of the project, and preferable beyond project end. Although there are data on some comparison communities, the number of comparison communities is low and their consistency is weak. This undermines the validity of the results presented.

• The data were collected by various organizations over the survey’s lifetime and there may be methodological differences in collection protocols between years that will need to be identified and taken into account.

• Data were collected on treatment communities benefiting from OTI interventions as well as control communities as defined by OTI, creating the opportunity to evaluate the impact of the activity through a comparison of the perceptions of respondents in the treatment and control communities.

2.2 ANALYSIS QUESTION

The two-part question that informs the analysis is as follows:

Is there a statistical difference in key indicators between treatment and control communities each year, or over the course of the five-year Program?

3 ACTIVITY BACKGROUND

3.1 CONTEXT

According to Program documents, the homicide rate in Honduras continues to drop, moving from 85 per100,000 in 2011 to around 39.13 per100,000 in 2017 - but is still far above the rate of 10 per100,000 that the UN uses to categorize violence as an “epidemic”. Violent crime in Honduras is primarily a phenomenon of low-income, densely-populated, urban areas – along with narco-trafficking routes along the Caribbean coast and the borders with Guatemala and El Salvador. This violence is a major driver of illegal immigration from Honduras to the US.

As part of the USAID/Honduras Country Development Cooperation Strategy goal to counter this violence and to promote a more prosperous and safer Honduras, focused its activities in five municipalities: Tegucigalpa, San Pedro Sula, Choloma, La Ceiba and Tela. These cities have a very high homicide rate. The average homicide rate in these municipalities between 2013 and 2017 ranges between 62.4 per 100,000 in Tegucigalpa to132. 6 per100,000 in San Pedro Sula (Table 1).

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 5

Table 1. Homicide rate per 100,00 inhabitants per targeted municipality per year

Municipality/ homicides rate/year

2013 2014 2015 2016 2017 Average

Country level 77.45 69.86 60.2 59.05 39.13 57.06

Tegucigalpa 83.37 60.2 66.3 39.6 62.4

San Pedro Sula 176.71 124.2 92.2 91.3 45.8 132.6

La Ceiba 143.97 93.3 89.2 105.4 52.1 121.0

Tela 85.78 74.2 57.4 48.7 41.2 76.8

Choloma 61.9 60.3 71.6 63.2 64.3

Source: SEPOL Honduras 2017 and INE Projected Population

Figure 1 and Figure 2 show the trends of the number of homicides and homicide rates at national and municipality level.

Given these epidemic rates of violence, OTI framed the following problem statement:

Violent crimes (and general lawlessness) primarily occur where the state does not exercise control over the use of force. The most violent areas of Honduran cities coincide with those areas where the security and justice sectors are least present, least effective, and most distrusted. Nationwide, the security and justice sectors are hampered by ineffectiveness, corruption, and insufficient resources to seriously combat an impunity rate estimated to be around 98 percent.

Figure 1. Homicide rate at national level: 2013–2017 Figure 2. Homicide rate by municipality: 2013–2017

77.45

69.86 60.2

59.05

39.13

2013 2014 2015 2016 2017

2013 2014 2015 2016 2017 Country level Tegucigalpa La Ceiba Tela Choloma San Pedro Sula

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 6

To address this problem, the Program had the following objectives, methodology, and scope:

Objective: Interrupt the systems, behavior, and perceptions that support gang control and accompanying violent crime.

● Sub-Objective 1: Empower citizens and institutions to act together and cooperatively increase security in targeted communities and sectors.

● Sub-Objective 2: Increase citizen confidence in and performance of government institutions, especially the police.

(Note: The two sub-objectives above comprise a reinforcing feedback loop: Increased security in targeted communities contributes to increased citizen confidence in government institutions, especially the police, which contributes to increased security.)

Geographic Areas: Urban neighborhoods of the cities of Tegucigalpa, San Pedro Sula, La Ceiba, and Tela that are strategically important for the gangs and for the police.

Overview of Program Methodology

● In pursuit of program objectives, activities are strategically sequenced to:

◦ First: Gain soft entry into communities - identify partners and build trust

◦ Second: Work concurrently with community (individual or group) actors and the police

• Strengthen community capacity to organize, mobilize, and act (including lobbying local government and security sector institutions, especially the police).

• Strengthen police capacity to engage with and join forces with the community.

◦ Third: Strengthen community cohesion, the union of community actors -including the police – working together to plan and organize for collective community security (which limits gang control).

Target Actors:

● Non- governmental actors such as parent-teacher associations, churches, youth groups, CSOs, and the private sector – with special emphasis on identifying and nurturing existing and emerging leadership.

● Governmental actors at the local level (e.g. municipal government, community councils, schools, clinics, and the police) and at the national level (e.g. Department of Education, National Electric Company, National Anti-Extortion Force, National Training Institute, Attorney General’s Office, and National Police).

Assumptions

● The government will continue to invest in the security and justice sectors.

● Strategies will be implemented for reform of the police (including implementation of the community police model), the Attorney General’s Office, and the court system.

● Communities will remain willing to take the gamble of allying with the police in a joint approach to improving security.

● Success, even small success, breeds additional and larger successes.

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 7

Definition of Success The Program measures whether it has effectively made progress toward its objectives by collecting data and performing regular analysis of target communities. This includes monitoring the number of active businesses and state institutions, homicide rates, perceptions of police performance and victimization, and active community partnerships.

The program uses a “stop-light” system to track progress along a handful of indicators that reflect changes in the security environment of target communities. Success for the program is when the majority of the indicators are green. This does not represent a total lack of insecurity, but rather strong momentum towards both continued improvement while maintaining security gains. This is when the Program “graduates” the community.

OTI works closely with INL and Civil Affairs where geographic target areas overlap. This collaboration is especially close in San Pedro Sula where the USAID Mission, INL, and Civil Affairs work together to implement a Place-Based Strategy (PBS). The PBS aligns efforts in three parts of the city with particularly strong gangs and associated high violent crime.

Overall Theory of Change

When communities trust their local police and hold them accountable for improving security, those communities and police will be active partners in disrupting and preventing crime and violence.

Figure 3. Honduras Convive! Objectives

Source: USAID

Primary Objective Interrupt systems, behaviors and perceptions that support illicit control and criminal violence in the territories

Objective 1 Increase security of communities in target geographic locations and sectors

Objective 2 Increase citizen trust in government institutions

1.1 Decreased control of territory by

illicit groups

1.2 Improved perception of safety in

affected zones

1.3 Strengthened community

cohesion

2.1 Strengthened National Police

2.2 Strengthened Criminal Justice

System

2.3 Increase in presence of state

institutions in communities

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 8

4 METHODS & LIMITATIONS

4.1 DATA

The data were collected between 2013 and 2017. In 2013, there were two data collections: in April and October. The data collection in April was conducted before any Convive activity started. Thus, a total of six surveys were included in the analysis. The methodology and results of each data collection are available from OIT/Convive. We describe in this section the steps taken to create the data set used for this report.

Each survey included a set of three types of communities: intervention, expansion, and control.

Intervention communities are those in which Convive was an active program at the time of the survey. The expansion communities were going to be part of Convive in the next 6 to 12 months, and control communities were neither intervention nor expansion communities, but have similar levels of violence. For the purpose of this analysis, both control and expansion communities are grouped together and referred to as the “comparison group,” as both types of communities did not have Convive at the time of survey.

Each round of data collection included previously surveyed communities and new communities.

For those communities that remained in the sample from one year to another, their status as treatment or control could have changed. We pooled all those data into one file. We created a unique code for each community and a status on whether the community was in the comparison or treatment group on that year. The complete list of communities by year and status can be found in Appendix I, Table 1. The result of the renewal sampling of communities and changing status resulted in the fact that the longitudinal sample of communities grew smaller at each data collection, and the number of intervention communities grew disproportionally greater than that of control communities (see the cells in red in Table 2).

Table 2. Number of communities by municipalities, sample type, treatment and control: 2013–2017

Municipalities/Year T/C

T/C

T/C

T/C

T/C

T/C

Tegucigalpa (all communities) 7/8 7/11 12/12 19/15 25/6 32/12

Longitudinal 6/7 6/7 8/5 9/2 11/2 10/0

SPS (full sample) 4/6 4/9 10/12 17/19 35/6 37/2

Longitudinal 4/6 4/6 7/3 8/2 10/0 8/0

La Ceiba 3/2 3/2 16/3 9/2 21/4 22/0

Tela 0/0 0/0 0/0 0/4 4/2 10/0

T: Treatment C: Control Disproportional number of treatment and control communities High disproportion of treatment to control communities

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 9

The previous table shows in red the communities where there are disproportionally more treatment than control communities. The disproportion between treatment and control communities occurs also in the non-longitudinal sample. Due to the smaller sample of communities through time, La Ceiba and Tela were excluded from the longitudinal analysis.

4.2 VARIABLES

In agreement with OTI/Convive, the analysis in this report focuses in three main themes:

1. Victimization

2. Security changes in the community

3. Perception of the performance of police in the community

In addition to these three themes, we analyzed two more questions that were consistent over time: major problems in the community and main violent actors in the community.

The victimization question had some changes over time. During the first two data collections in 2013, the question did not have a time reference. Thus, the question was: “Have you or a family member been victim of a violent crime?” In the rest of the questionnaires, the prompt refers to the last six months: “In the last six months, have you or a family member been a victim of a violent crime?” This difference makes the percentage on positive responses higher for the first two surveys.

The question about the main problems in the community is a multiple-choice question. We analyzed what the main answers were to reduce the number of choices to four: drugs/drug trafficking, violence, robbery, and gangs. All of these options were available in all questionnaires, with the exception of gangs. This option was not available for the third survey in 2014.

The question about the main groups that cause problems in the community were also multiple-choice. After examining the answers, we selected the two most common responses: gang members and thieves.

The question on security change in the community in the last six months had some differences between the first rounds of the survey and the rest. The answers for this question were on a 5-point Likert scale that had the following options: “it has improved a lot”, “it has improved a little”, “it is the same”, “it has worsened a little”, and “it has worsened a lot”. However, in the first two questionnaires, the last two options were “it has reduced a little” and “it has reduced a lot”, meaning the security situation has gotten worse. In addition, in the first questionnaire the question prompts the respondent to compare the situation compared to the previous year and not last six months. The Likert scale value was kept on a 5-point value, but these differences are noted in the results section.

Finally, the question about how the respondent rates the performance of the national police to fight crime in their community was available for all questionnaires. The answer to this question was also a 5-point Likert scale that has the following options: “very bad”, “bad”, “regular”, “good”, and “very good”. In the third survey (2014), the scale was a 4-point scale without the option “regular” available in the other rounds. That scale was realigned with the other scale.

In addition to these variables, we also looked for control variables that were constant over all the questionnaires. We found that sex and age groups were available, and they are included in

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 10 the models below. The homicide rates for each community were included in the model as a control variable, but because of their lack of statistical significance they were dropped from the final analysis. The homicide rates for each municipality are discussed on the previous section.

(A full review of the questionnaires used over the course of the Program and recommendations for future use are available in Appendix II.)

4.3 ESTIMATION APPROACH

The analysis of the surveys required consideration of two distinct characteristics: the sample design of each survey, and the aggregation over time. These two characteristics have important effects on the statistical properties of each indicator.

As we understand the sampling process, individuals were drawn randomly from intervention, expansion and control communities. The samples were drawn from each of the following municipalities: Tegucigalpa, San Pedro Sula, La Ceiba and Tela. This last municipality, Tela, was included in the sample starting in 2015. Two other municipalities were included in one survey:

La Lima in 2013 and Choloma in 2017. Choloma was chosen as a control to other San Pedro Sula communities, so it entered as a control community. These two municipalities were excluded from the analysis since no longitudinal information is available.

Hierarchical Linear Modeling (HLM) was used for econometric analysis, however the resulting models did not fit the data well (resulting discrepancies in slope of trends that presented inaccuracies in interpretation). It was determined that the multi-year graphs, with notes on significant differences between treatment and control in each year, was the most appropriate representation of the data for the purposes of this report. While the results are simple means, the standard errors used to determine whether differences are statistically significant take into account the cluster of individuals within communities.

4.4 LIMITATIONS

This analysis has several limitations from the impact evaluation perspective. The results are undermined by problems in the consistency of the questions, changes in the sample of communities and their treatment and comparison status, and selection of the comparison communities. All of these issues affect the internal and external validity of the results. Internal validity refers to the reliability of the results we found. This means that the differences we found cannot be attributable to Convive in a strict sense. We did control for as many variables available, but the selection process of treatment and control communities, and the length of time examined allows for the population to adjust and dilute the results.

The external validity is the capacity to generalize the results to a greater population. To achieve this, the results should be representative of a population and chosen in a deliberate manner to represent a population. These two aspects are lacking given the sample selection of both respondents and communities.

Apr-13 Oct-13 2014 2015 2016 2017 Control 0.307147 0.720086 0.158116 0.166227 0.1121 0.081 Treatment 0.299207 0.720784 0.121372 0.152659 0.1011 0.059

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 11

5 RESULTS

This section presents the results from the analysis for each outcome variable. More detailed tables, the averages for each municipality, and results of the HLM analysis can be found in Appendix I.

Finally, for this study we set the level of statistical significance to a minimum of 5 percent.

When differences are noted, it means this level has been achieved, otherwise the means and differences in means are deemed to not be statistically significant.

5.1 VICTIMIZATION

The percentage of people that declared during the survey that they or a member of their family had been the victim of a violent crime in the last six months is presented in Figure 4. This percentage is higher in the first two surveys given that the wording of the question did not include a period of reference for the violent crime. All of the values are statistically significantly different from zero, but the differences between treatment and control are not.

Figure 4. Percentage of people that have been victim of a violent crime in the last 6 months: 2013–2017

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

Control Treatment

- Control Treatment Apr-13 24.2174 22.4946 Oct-13 37.0194 36.8789 2014 22.9605 35.5447 2015 35.6201 35.5632 2016 26.5113 28.1625 2017 16.7 11.7

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 12

5.2 PROBLEMS IN THE COMMUNITY

This section presents the percentage of people that state one or more of the following problems: drugs/drug trafficking, violence, gangs, and robbery. These problems are the most commonly mentioned in response to a multiple-choice question.

5.2.1 DRUGS/ DRUG TRAFFICKING

This indicator shows the percentage of people that acknowledged during the survey that drugs or drug trafficking is one of problems that affects the community. All of the values are statistically significantly different from zero, but the differences between treatment and control are not, except for 2014. The perception of drugs in 2017 dropped below that of 2013 in both treatment and control communities.

Figure 5:. Percentage of people that identify drugs/drug trafficking to be a problem in their community: 2013–2017

- Comparison Treatment 13-Apr 8.91908 7.78659 13-Oct 23.9741 19.5241 2014 49.0328 55.6351 2015 26.4732 34.3625 2016 26.5113 28.1625 2017 18.7 24.5

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 13

5.2.2 VIOLENCE

The violence indicator shows the percentage of people that mentioned during the survey that violence is one of problems that affects the community. All of the values are statistically significantly different from zero, and the differences between treatment and control are different in 2015 and 2017 (where control communities report less violence). The perception of violence has increased between 2013 and 2017, from less than 10 percent to around 20 percent.

Figure 6. Percentage of people that mention that violence is a problem in their community: 2013–2017

13-Apr 13-Oct 2014 2015 2016 2017

Comparison Treatment

13-Apr 25.2806 21.4852 13-Oct 60.1296 46.746 2014 - - 2015 22.3395 32.1898 2016 32.8478 30.4709 2017 0.7 0.6

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 14

5.2.3 GANGS

This indicator shows the percentage of people that mentioned during the survey that gangs are one of the problems that affect the community. All of the values are statistically significantly different from zero, but the differences between treatment and control are not, except for 2015.

The perception that gangs are a problem increased between 2013 and 2017, from less than 10 percent to around 20 percent. In 2014 “gangs” was not an option in the responses, so it is not possible to report a value for that year.

The percentage for 2017 is surprisingly low. There are no changes in Honduras that could explain such a level. While the explanation of this issue is beyond the scope of this report, the low value might be a result of survey implementation.

Figure 4. Percentage of people that mention that gangs are a problem in their community: 2013–2017

13-Apr 27.5251 30.4254 13-Oct 28.2937 25.7523 2014 32.0017 31.2476 2015 35.8839 30.5889 2016 28.1865 32.3638 2017 15.6 21.8

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 15

5.2.4 ROBBERY

This indicator shows the percentage of people that declared during the survey that robbery and mugging are one of problems that affect the community. All the values are statistically significantly different from zero, but the differences between treatment and control are not.

The perception of robbery decreased between 2013 and 2017 in both treatment and control communities.

Figure 8. Percentage of people that mention that robbery is a problem in their community: 2013–2017

13-Apr 52.3331 54.8666 13-Oct 71.1015 56.8929 2014 56.6863 56.7659 2015 42.4802 50.4288 2016 44.4283 44.7368 2017 13.7 18.6

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 16

5.3 PERPETRATORS OF VIOLENCE

In all survey waves, people provided their opinion about which groups that cause problems in their community. The responses for each year were analyzed, and we determined that only two options were common: gang members and thieves.

5.3.1 GANG MEMBERS

This indicator shows the percentage of people that stated that gang members are troublemakers in their community. All of the values are statistically significantly different from zero, but the differences between treatment and control are not, except for the second survey.

The perception that gang members are troublemakers decreased between 2013 and 2017, from over 50 percent to less than 20 percent in both treatment and control communities.

Figure 9. Percentage of people that mention that gang members are troublemakers in their community: 2013–2017

13-Apr 12.9356 12.0404 13-Oct 47.5162 50.5948 2014 26.4508 22.2767 2015 37.467 29.6741 2016 25.9286 30.5633 2017 28.5 32.9

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 17

5.3.2 THIEVES

This indicator shows the percentage of people that stated that thieves are one of the troublemakers in their community. All of the values are statistically significantly different from zero, but the differences between treatment and control are not. The perception that thieves are one of the main troublemakers has increase between 2013 and 2017, from over 12 percent to over 25 percent.

Figure 10. Percentage of people that mention that thieves are troublemakers in their community: 2013–2017

13-Apr -1.205882 -1.139985 13-Oct -0.2485876 -0.3002813 2014 0.5573555 0.562041 2015 0.5067024 0.8291836 2016 0.459897 0.5602606 2017 0.5 0.6

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 18

5.4 CHANGE IN PERCEPTION OF SECURITY IN THE COMMUNITY

The results on the perception of how much security has changed in the last six months show an increase in perception that the situation is improving (Figure ). The difference between treatment and control communities was statistically significant in 2015 and 2017. Communities in the treatment group perceived those years that security had improved more than those in the control group.

Figure 11. Average score on how much security in the community has changed in the last six months: 2013–2017

5.5 OPINION ON NATIONAL POLICE PERFORMANCE IN THE COMMUNITY

The results on perception of the performance of the police in fighting crime in the community converges to a mediocre performance (Figure 5). There are no statistically significant differences between treatment and control communities. The performance as seen by the community seems to regress towards the median of the scale (“regular”) after improving in earlier in 2014 and 2015.

-1.5

-1

-0.5

0.5

13-Apr 13-Oct 2014 2015 2016 2017

Control Treatment

Im pr ov in g W or se ni ng

13-Apr -1.205882 -1.139985 13-Oct -0.2485876 -0.3002813 2014 0.5573555 0.562041 2015 0.5067024 0.8291836 2016 -0.002913329 0.102954755 2017 0.2 0.3

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 19

6 KEY FINDINGS AND CONCLUSIONS

Key Findings

• Homicide rates dropped in all municipalities from 2013-2017 with the exception of Choloma.

• Reported victimization also decreased in all communities (Convive and comparison) from 2013-2017.

• The perception of frequency of drug use, drug trafficking, violence, and gangs decreased in Convive communities at a faster rate than comparison communities from 2015-2017, though comparison communities were beginning to catch up toward 2017. These perceived improvements took place even though the homicide rate leveled off or increased slightly from 2015-2016 in some communities.

• Perceptions of security improvements were modestly better in Convive communities than in treatment communities from 2013-2017, with the greatest improvement in Convive communities compared to comparison communities in 2015 and 2017. These perceived improvements may have been bolstered by decreased homicides in all included communities (except those in La Ceiba) from 2013-2015.

• Perception of police performance increased slightly in all communities (Convive and comparison) from 2013-2017.

Conclusions

This report examines the results obtained through the annual survey on victimization and perception in the Convive program. Each survey sampled different communities over time and included some changes to the questions and answers. We identified questions and communities that could be compared over time in three areas: victimization, security change and perception of police performance. The analysis included two samples: one with all the communities interviewed in each year, and another with only those communities that were sampled every year between 2013 and 2016. We conducted a comparison between two types of communities: one where Convive was implemented, and others where it was not. This comparison was by looking at differences in simple means.

Figure 5. Average score on police performance in fighting crime in the community: 2013–2017

-1.5

-1

-0.5

0.5

13-Apr 13-Oct 2014 2015 2016 2017

Control Treatment

Go od

Po or

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 20

For victimization, we analyzed whether the person or family members have been victims of a violent crime in the last six months, the main problems in the community (violence, drugs, gangs, and robbery), and the main troublemakers in the community (thieves and gangs). There was no difference between treatment and comparison on victimization, although the proportion of people that have been victim of a violent crime has decreased over time. There are differences between treatment and comparison communities on the main problems in the community. Treatment communities have a lower level of problems reported (drugs, gangs, and robbery), but also a slower rate of improvement (drugs, gangs and robbery) after starting at the same position in April 2013 which implies that comparison communities are catching up to the lower levels achieved by treatment communities. The percentage of people that think that gangs and thieves are the main troublemakers in the community is also decreasing over time.

Treatment communities have a lower percentage, but they follow the same trend as comparison communities over time.

The perception on whether the security in the community has improved increased over time, but there is no difference between treatment and comparison communities.

Finally, perceptions on the performance of the National Police increased from low levels to a “regular” level over time. However, there is no difference between treatment and comparison communities.

These results need to be understood within the dynamic Honduran context, where comparison communities are not isolated from national policies or the influence of other development programs conducted by other organizations. In addition, it is clear that when a community improves, it might attract new people, and other communities might try to imitate the programs of successful communities. This dilutes the possibility of measure a “pure” impact of Convive.

These surveys were never intended to be used as part of an impact evaluation. They were originally used to monitor the situation in the communities. Comparison communities sampled were thought of as future communities for Convive. This also contributes to difficulty to determine the impact of Convive.

In addition to this, the change in sample, questions, answers, communities and consistency on data collection contributes to the difficulty of conducting a proper impact evaluation.

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 21

7 APPENDIX I: TABLES AND HLM RESULTS

This page intentionally left blank.

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 22

Appendix I Table 1. Communities included in each sample by municipality, year and status: 2013–2017

Year Municipalities and Community Apr

Oct

2014 2015 2016 2017

La Ceiba

Col. Altos de Sinaí

I I I I Col. Ángel Salinas

I

I I

Col. Bautista

Col. Bonitillo C C I I I I Brisas del Atlántico

Brisas del Caribe

Brisas del Mar

Col. Girasoles

Col. Militar

Col. Municipal

Riveras del Caribe

Col. El Confite C C I I I I Col. Las Mercedes

C C I I

Col. San Judas/San José I I C

Búfalo

C C C

Col. Primero de Mayo I I I

Col. Suyapa

C

Col. San Juan

Col. San Judas Tadeo

1 de Mayo

Sector Bonitillo

Tegucigalpa

Col. 15 de Septiembre

Col. San Miguel I I I I I I Col. Villa Cristina C C C I I I Col. Villa Franca C C C I I I Col. Villa Vieja

C C

La Flor #1

C I I I I La Flor # 2

Villanueva

C I I Col. Oscar A. Flores

Col.30 de Noviembre

C I I I Col. 3 de Mayo C C I

Suazo Córdova

Col. Altos de la Travesía C C C I I

Col. Ayestas

I I I Col. Campo Cielo I I I I I I Col. Canaán

Col. Divanna

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 23

Year Municipalities and Community Apr

Oct

Col. El Pedregal de San José

Col. El Sitio I I I I I I Col. Estados Unidos I I I I I I Col. Francisco Morazán

Col. Peña

Col. La Sosa I I I I I I Col. La Travesía I I I

Col. Los Pinos

Col. Modesto Rodas Alvarado

Col. Mololoa C C C C C

Col. Nueva Capital

Col. La Era

I I I I Col. Nueva Suyapa

Col. Quebrachitos C C C C C

San Martin C C I I I I Bo. El Pastel

C I

Col. Zapote

Col. 14 de Febrero

Col. Altos de la Sosa C C

La Trinidad I I C

1 de Diciembre

14 de Marzo

21 de Febrero

Carpintero

Carrizal 1

Chile

Faldas del Pedregal

Flor del Campo

La Australia

La Sagastume

La Vega

Las Torres

Providencia

El Pananal

United States of America

Villa Union

Tela

Bo. El Centro

Bo. El Paraíso

Col. 4 de Enero

Col. La Grant

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 24

Municipalities and Community Apr

Oct

Col. San Alejo

C C I Las Brisas

San Juan

Tornabe

Venecia

San Pedro Sula

Col. Callejas

Col. Cerrito Lindo

Col. Lomas del Carmen

Col. Alfonzo Lacayo

Col. Satélite I Etapa

Col. Satélite II Etapa

Col. Satélite III Etapa

Pradera

La Ceiba

Col. 10 de Septiembre

Barrio Cabañas

Col. 14 de Julio

Col. Ebenezer

Col. Los Zorzales

Col. Chamelecón

Col. Los Laureles

Col. Losusa

Col. Morales I

Col. Palmira C C C I I I Col. Sabillón Cruz

Col. San Isidro I I I I I I Col. San Juan/ San Jorge I I I I I I Col. Terencio Sierra C C C I I I Col. Santa Ana I I I I I I Col. 6 de Mayo C C I I I I Col. Rivera Hernández I I I I I I Col. Sinaí C C I I I

Col. Morales III C C C C I

Col. Suyapa C C I C I I Col. San Antonio

Col. 6 de Mayo Asentamientos Humanos (Bo.)

Col. Brisas del Sur

Col. Bloque Central

Col. Centralista

Col. Planeta

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 25

Municipalities and Community Apr

Oct

Bo. Concepcion

Bo. Medina

Brisas del Sauce

Colonia Independencia

El Centro

La Celio

La Montana

Reparto Lempira

Rio de Piedras

Trejo

La Lima

La Planeta

Choloma

La Jutosa

I: Intervención; C: Comparison

Appendix II Table 2. Percentage of individuals that report being victims of violence, by year: 2013–2017

Year Treatment Control Difference Apr-20131 29.9 30.7 -0.8 Oct-20131 72.1 72.0 0.1 2014 12.1 15.8 -3.7 2015 15.3 16.6 -1.4 2016 10.1 11.2 -1.1 2017 5.4 8.0 -2.7

* p<0.05 1The question does not have a time reference for when the violent act was experienced.

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 26

Appendix I Table 3. Percentage of individuals that report main problems in their community, by year and problems: 2013–2017

Drugs Violence Gangs Robbery

Treat ment

Cont rol

Differ ence Treat ment Cont rol Differ ence Treat ment

Cont rol

Differ ence

Treat ment

Cont rol

Differ ence

Apr- 13 22.5 24.2 -1.7 7.8 8.9 -1.1

21.5 25.3 -3.8

30.4 27.5 2.9

Oct- 13 36.9 37.0 -0.1 19.5 24.0 -4.4

46.7 60.1 -13.4

25.8 28.3 -2.5

2014 35.5 23.0 12.6* 55.6 49.0 6.6 — — — 31.2 32.0 -0.8

2015 35.6 35.6 -0.1 34.4 26.5 7.9* 32.2 22.3 9.9* 30.6 35.9 -5.3

2016 28.2 26.5 1.7 24.5 21.2 3.3 30.5 32.8 -2.4 32.4 28.2 4.2

2017 11.7 16.7 -5.0 24.5 18.7 5.8* 0.6 0.7 -0.1 21.8 15.6 6.2

— Not Available. Data were not collected

* p<0.05

Appendix I Table 4. Percentage of individuals that identify troublemakers in their community, by year and groups: 2013–2017

Gang members Thieves Treatment Control Difference Treatment Control Difference Apr-13 54.9 52.3 2.5 12.0 12.9 -0.9 Oct-13 56.9 71.1 -14.2* 50.6 47.5 3.1 2014 56.8 56.7 0.1 22.3 26.5 -4.2 2015 50.4 42.5 7.9 29.7 37.5 -7.8 2016 44.7 44.4 0.3 30.6 25.9 4.6 2017 18.6 13.7 4.9 32.9 28.5 4.4

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 27

Appendix I Table 5. Average perception on the security and the police in the community, by year: 2013–2017

Security1 National Police2 Treatment Control Difference Treatment Control Difference Apr-13 -1.1 -1.2 0.1 -1.3 -1.2 -0.1 Oct-13 -0.3 -0.2 -0.1 -0.4 -0.3 -0.1 2014 0.6 0.6 # 0.5 0.6 -0.1 2015 0.8 0.5 0.3* 0.2 0.1 0.1 2016 0.6 0.5 0.1 0.1 # 0.1 2017 0.6 0.5 0.2* 0.3 0.2 0.1

# Rounds to zero

* p<0.05 1The lower ends of the scale have different wording. Instead of "It has worsen a little" or "It has worsen a lot", the options are "It has reduced a little" or "It has reduced a lot".

2In 2014, the option "Regular" was not available. That scale was re-aligned to match the other years.

NOTE: this is a 5-point scale centered at zero for regular. Positive values represent positive views while negative represent negative views.

Appendix I Table 6. Percentage of individuals that report being victims of violence, by year and municipio: 2013–2017

La Ceiba Tegucigalpa Tela San Pedro Sula

Year Treatm ent

Cont rol

Differ ence Treat ment Cont rol Differ ence

Tre at me nt

Cont rol

Differ ence Treat ment Cont rol Differ ence

Apr- 20131 31.8 34.4 -2.6 26.8 27.3 -0.5 — — — 32.8 32.6 0.2

Oct 20131 61 65.1 -4.1 88.9 93.9 -5 — — — 53.3 53.4 -0.2

2014 6.2 1.9 4.2* 10.9 9.5 1.4 — — — 15.6 22.0 -6.4 2015 15.0 9.8 5.3 18.3 17.0 1.3 † † † 12.3 18.7 -6.4* 2016 7.7 7.1 0.6 13.1 15.7 -2.7 7.7 7.7 # 10.2 12.3 -2.1

2017 5.5 † † 9.6 11.1 -1.5 4.2 † † 4.0 2.5 1.5

— Not Available. Data were not collected † Not applicable. No data on control communities # Rounds to zero

1The question does not have a time reference for when the violent act was experienced.

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 28

Appendix I Table 7. Percentage of individuals that report that drugs is a main problem in their community, by year and municipio: 2013–2017

Appendix I Table 8. Percentage of individuals that report that violence is a main problem in their community, by year and municipio: 2013–2017

Percentage of individuals that report that drugs is a main problem in their community, by year and municipio:2013–2017

Year Treatment Control Difference Treatment Control Difference Treatment Control Difference Treatment Control Difference

Apr-13 11.4* 15.6 -4.2 30.2* 25.5* 4.7 — — — 18.9* 26.4* -7.5 Oct-13 26.4 44.5 -18.1 43.3* 57.0* -13.7* — — — 33.9 16.2 17.7 2014 9.2* 7.3 2 47.4* 34.0* 13.4 — — — 25.2* 21.1* 4.1 2015 26.7* 26.0* 0.7 50.0* 43.8* 6.3 † † † 28.6* 35.4* -6.8 2016 18.7* 17.3* 1.4 43.8* 34.6* 9.2 19.8* 24.7 -4.9 26.8* 27.2* -0.4 2017 21.85 — — 18.5 19.9 -1.3 19.8 † † 7.2 10.8 -3.6

La Ceiba Tegucigalpa Tela San Pedro Sula

— Not Available. Data were not collected † Not applicable. No data on comparison communities

* p<0.05

Percentage of individuals that report that violence is a main problem in their community, by year and municipio:2013–2017

Year Treatment Control Difference Treatment Control Difference Treatment Control Difference Treatment Control Difference

Apr-13 7.1 17 -9.9* 8.9 5.3* 3.7 — — — 6.7 9.3* -2.7 Oct-13 16.6 34.2* -17.6* 11.9* 19.3* -7.4 — — — 33.4 25.5* 7.9 2014 38.1* 19.5 18.6 60.3* 62.2* -1.9 — — — 53.5* 52.1* 1.4 2015 33.7* 22 11.8 31.0* 24.5* 6.5 † † † 38.3* 34.9* 3.4 2016 15.1* 17 -1.8 35.1* 23.3* 11.8* 6.5 10.1 -3.6 27.1* 29.8* -2.7 2017 22.9 23.8 16 7.8* 19.8 † † 25.4 23.8 1.5

La Ceiba Tegucigalpa Tela San Pedro Sula

— Not Available. Data were not collected † Not applicable. No data on comparison communities

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 29

Appendix I Table 9. Percentage of individuals that report that gangs is a main problem in their community, by year and municipio: 2013, 2015–2017

Appendix I Table 10. Percentage of individuals that report that robbery is a main problem in their community, by year and municipio: 2013–2017

Percentage of individuals that report that gangs is a main problem in their community, by year and municipio:2013, 2015–2017

Year Treatment Control Difference Treatment Control Difference Treatment Control Difference Treatment Control Difference

Apr-13 13.2 29.4 -16.2 24.8* 20.6* 4.1 — — — 22.0* 28.2* -6.2 Oct-13 37.6 50.0* -12.4 51.8* 75.6* -23.8* — — — 44.9* 48.5* -3.5 2015 23.4 22 1.5 28.1* 19.0* 9.1 — — — 43.9* 28.4* 15.5* 2016 23.7* 28.0* -4.2 32.8* 28.5* 4.3 12.1* 16.6 -4.5 35.5* 52.2* -16.7* 2017 — — — 1.32 1.12 0.2 0.64 † † 0.41 # 0.4*

La Ceiba Tegucigalpa Tela San Pedro Sula

— Not Available. Data were not collected † Not applicable. No data on comparison communities # Rounds to zero

* p<0.05

Percentage of individuals that report that robbery is a main problem in their community, by year and municipio:2013–2017

Year Treatment Control Difference Treatment Control Difference Treatment Control Difference Treatment Control Difference

Apr-13 43.9 33.7 10.2 28 16.4* 11.6 — — — 25.8* 36.2* -10.4 Oct-13 37.3 50.7* -13.4 24.2* 34.9* -10.8 — — — 20.5 15.8 4.7 2014 31.9* 17.6 14.4 36.7* 36.5* 0.2 — — — 23.0* 35.1* -12.1 2015 40.8 26.8* 13.9 31.3* 40.8* -9.4 † † † 21.1* 38.9* -17.9* 2016 25.5* 21.1 4.4 40.1* 37.3* 2.8 18.2* 15.4 2.8 34.5* 32.9* 1.6 2017 13.4 — — 10.1 13.31 -3.21 † † † 28.1 20.1 8

La Ceiba Tegucigalpa Tela San Pedro Sula

— Not Available. Data were not collected

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 30

Appendix I Table 11. Percentage of individuals that identify gang members as the trouble-makers in their community, by year and municipio: 2013–2017

Percentage of individuals that identify gangmembers as the trouble-makers in their community, by year and municipio:2013–2017

Year Treatment Control Difference Treatment Control Difference Treatment Control Difference Treatment Control Difference

Apr-13 46.4 48.2 -1.8 48.6* 48.2* 0.3 — — — 68.0* 58.1* 9.9 Oct-13 47.1 58.9 -11.8 54.1* 76.0* -21.9* — — — 67.7* 69.9* -2.2 2014 26.9* 18.3 8.6 53.3* 53.7* -0.4 — — — 69.8* 70.1* -0.2 2015 35.6* 47.2 -11.5 52.9* 38.0* 14.9* † † † 60.7* 56.9* 3.8 2016 32.3* 33.6* -1.3 54.8* 46.7* 8.1 12.1* 20.6 -8.5 51.6* 66.8* -15.3* 2017 19 — — 14.2 12.6 1.5 22.7 † † 19.7 15.9 3.8

La Ceiba Tegucigalpa Tela San Pedro Sula

— Not Available. Data were not collected

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 31

Appendix I Table 12. Percentage of individuals that identify thieves as the trouble-makers in their community, by year and municipio: 2013–2017

Appendix I Table 13. Average perception on how much the community has improved, by year and municipio: 2013–2017

Percentage of individuals that identify robbers as the trouble-makers in their community, by year and municipio:2013–2017

Year Treatment Control Difference Treatment Control Difference Treatment Control Difference Treatment Control Difference

Apr-13 8.2 11 -2.8 12.1* 6.8* 5.3* — — — 14.1* 19.8* -5.6 Oct-13 51.2* 58.6 -7.4 52.2* 64.6* -12.3 — — — 47.6 28.4* 19.2 2014 19.2* 14.5* 4.7 21.5* 23.4* -1.9 — — — 24.2* 33.1* -8.9 2015 37.0* 19.5 17.4 36.4* 45.8* -9.4 † † † 16.6* 40.6* -24.0* 2016 25.2* 18.2* 7.1 43.3* 39.1* 4.3 16.6* 15.4* 1.2 30.3* 25.6* 4.7 2017 21.6 — — 23.5 20.3* 3.2 10.1 † † 39.6 43.9 -4.3

— Not Available. Data were not collected † Not applicable. No data on comparison communities

* p<0.05

La Ceiba Tegucigalpa Tela San Pedro Sula

Year Treatment Control Difference Treatment Control Difference Treatment Control Difference Treatment Control Difference

Apr-20131 -1.2* -1.3* # -1.2* -1.2* # — — — -1.0* -1.2* 0.2

Oct 20131 -0.1 -0.1 # -0.4* -0.3* # — — — -0.3 -0.2 -0.1 2014 1.4* 1.1* 0.3* 0.1* 0.3 -0.2 — — — 1.0* 0.8* 0.1 2015 1.3* 0.9 0.3 0.4* 0.3* 0.1 † † † 0.9* 0.5* 0.4* 2016 1.1* 0.8* 0.3* 0.1 0.2 -0.1 0.7* 0.7* # 0.5* 0.3 0.2 2017 — — — 0.7* 0.5* 0.2* — — — 0.4* 0.5* -0.1

La Ceiba Tegucigalpa Tela San Pedro Sula

— Not Available. Data were not collected † Not applicable. No data on comparison communities # Rounds to zero

* p<0.05 1The lower ends of the scale have different wording. Instead of "It has worsen a little" or "It has worsen a lot", the options are "It has reduced a little" or "It has reduced a lot".

NOTE: this is a 5-point scale centered at zero for regular. Positive values represent positive opinions, while negative represent bad images.

Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 32

Appendix I Table 14. Average perception on the police performance in the community, by year and municipio: 2013–2017

Year Treatment Control Difference Treatment Control Difference Treatment Control Difference Treatment Control Difference

Apr-20131 -1.1 -1.1* # -1.4* -1.2* -0.2* — — — -1.4* -1.3* -0.1

Oct 20131 -0.3 -0.3 # -0.4* -0.4* # — — — -0.5* -0.2 -0.3 20142 1.1* 0.6 0.5 0.2* 0.2 # — — — 0.9* 0.8* # 2015 0.6 0.3 0.3 -0.1 # -0.1 † † † 0.2* 0.1* 0.1 2016 0.5* 0.2 0.2 -0.2* -0.1 # 0.3* 0.1 0.2 # -0.1* 0.2* 2017 — — — 0.2* 0.3* -0.1 — — — 0.3* 0.3 -0.1

— Not Available. Data were not collected † Not applicable. No data on comparison communities # Rounds to zero

* p<0.05 1The lower ends of the scale have different wording. Instead of "It has worsen a little" or "It has worsen a lot", the options are "It has reduced a little" or "It has reduced a lot".

2In 2014, the option "Regular" was not available. The scales was re-aligned to match the other years.

NOTE: this is a 5-point scale centered at zero for regular. Positive values represent positive opinions, while negative…