Attachment J.23 MID-TERM PERFORMANCE EVALUATION CDLO.pdf

DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government

MID-TERM PERFORMANCE EVALUATION

OF COMMUNITY DEVELOPMENT FOR

LICIT OPPORTUNITIES (CDLO)

FINAL EVALUATION REPORT

July 2022

Contract Title: USAID/Colombia Monitoring, Evaluation and Learning Activity

Contract Number: 72051419C00001

Submitted: July 25, 2022

Contractor: Panagora Group, Inc.

COR: Omar López

Authors: Hugo Navarro (Director of the Evaluation), Allison Benson (Thematic Expert), Lucía Llanes Valenzuela (Evaluation Specialist), Lizeth Molina (Quantitative Specialist), and Pablo Gutiérrez (Qualitative Specialist). Members of MEL Activity: Orlando Gracia, Luisa Fernanda Cardona, Juan Guillermo Bedoya, Yazmín Medina, David Doncell, Javier Rodríguez, and Julieth Forero.

Panagora Group Contacts:

Ana María Rivera Chief of Party Carrera 7 No. 74B-36, Piso 6, Bogotá, Colombia Email: arivera@panagoracolombia.net

Orlando Gracia Lead Evaluation Expert Carrera 7 No. 74B-36, Piso 6, Bogotá, Colombia Email: ogracia@panagoracolombia.net

Majella van der Werf Program Director 8601 Georgia Avenue, 8th Floor, Silver Spring, MD, USA Email: majellavanderwerf@panagoragroup.net

(DELETE THIS BLANK PAGE AFTER CREATING PDF. IT’S HERE TO MAKE FACING PAGES AND

LEFT/RIGHT PAGE NUMBERS SEQUENCE CORRECTLY IN WORD. BE CAREFUL TO NOT DELETE

THIS SECTION BREAK EITHER, UNTIL AFTER YOU HAVE GENERATED A FINAL PDF. IT WILL

THROW OFF THE LEFT/RIGHT PAGE LAYOUT.)

i | COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION USAID.GOV

CONTENTS

ACRONYMS II

ABSTRACT III

EXECUTIVE SUMMARY 1

EVALUATION PURPOSE 6

EVALUATION QUESTIONS AND METHODOLOGY 10

EVALUATION FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 17

ANNEX I : RECOMMENDATIONS ACTIONS MATRIX 65

ANNEX II: SUPPLEMENTAL ANALYSIS AND DATA 78

ANNEX III: FINDINGS, CONCLUSIONS AND RECOMMENDATIONS

MATRIX 107

ANNEX IV: DETAILED METHODOLOGY 114

ANNEX V : INSTRUMENTS 171

ANNEX VI: DATA SOURCES 339

ANNEX VII: EVALUATION SCOPE OF WORK 352

ANNEX VIII: SUMMARY INFORMATION ABOUT EVALUATION

TEAM MEMBERS 370

ANNEX IX: SIGNED DISCLOSURES OF CONFLICTS OF INTEREST

FROM EVALUATION TEAM MEMBERS 371

USAID.GOV COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION | ii

ACRONYMS

ADS Automated Directives System

AMELP Activity Monitoring, Evaluation, and Learning Plan

ART Agency for Territorial Renewal (Agencia de Renovación del Territorio)

CBI Capacity Building Index (Índice de Fortalecimiento de Capacidades)

CBO Community-Based Organization

CDLO Community Development and Licit Opportunities Activity

CLA Collaboration, Learning, and Adapting

COP Colombian Peso

COR Contracting Officer’s Representative

DNP National Planning Department (Departamento Nacional de Planeación)

FARC Armed Forces of Colombia (Fuerzas Armadas Revolucionarias de Colombia)

GIS Geographic Information Systems

GOC Government of Colombia

IP Implementing Partner

IPSOS Institut Publique de Sondage de secteurs d'opinion

JAC Community Action Board (Junta de Acción Comunal)

MEL Monitoring, Evaluation, and Learning

OPI Organizational Performance Index

PDET Territorially Focused Development Program (Programas de Desarrollo con Enfoque Territorial)

PNIS Comprehensive National Program for the Substitution of Illicit Crops (Programa Nacional Integral de Sustitución de Cultivos ilícitos)

SENA National Learning Service (Servicio Nacional de Aprendizaje)

SOW Scope of Work

USAID U.S. Agency for International Development iii | COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION USAID.GOV

ABSTRACT

The United States Agency for International Development (USAID) Colombia manages the Community Development and Licit Opportunities Activity (CDLO). The Activity was initiated in August 2017 to build the capacity of organizations in rural communities affected by Colombia’s internal armed conflict, working with government and private actors to implement social and economic development initiatives.

The Monitoring, Evaluation, and Learning Activity (MEL) conducted a mid-term performance evaluation of CDLO to assess “the achievements of Community Development and Licit Opportunities Activity (CDLO) interventions, deliverables, and outcomes according to the work plan and theory of change and to what extent they are relevant based on the current national and local contexts.”1

The evaluation included closed, closing, and ongoing CDLO activities as of September 30, 2020.

Qualitative and quantitative information was collected between February 8 and August 5, 2021. The Implementing Partner (IP) provided information on sales (July 2021 cutoff), commercial alliances (August 2021 cutoff), and the Organizational Performance Index (OPI) (November 2021 cutoff). Implementing a mixed-methods approach, the evaluation found that CDLO has built networks of interdependence among community-based organizations (CBOs). These networks generated clusters of licit economic activities in areas with armed groups and incidences of illicit economic activities. CDLO has strengthened the capacities of CBOs by working with them to design and implement economic and infrastructure projects. CDLO´s support strengthened the organizational, productive, and commercial capacities of CBOs, including the construction of commercial partnerships. The evaluation also found that infrastructure projects executed jointly with Community Action Boards (JACs) responded to community needs and improved road infrastructure. These projects have been effective in strengthening the organizational capacities of CBOs, contributing to the efficient management of resources, the generation of surpluses to finance new projects, and CBOs’ legitimacy. As a result, these organizations are more prepared to execute future infrastructure projects. Moreover, the evaluation found that CDLO promoted the creation of dialogue networks with local actors and contributed to building trust and empowering communities to participate in planning and participatory spaces.

CDLO’s main challenges in the future include consolidating and expanding the commercial capacities of producer CBOs to further increase and sustain income generation. Other challenges include scaling and replicating the successful infrastructure project execution model with JACs. Implementing a robust strategy to systematize, socialize, and transfer the methodologies and lessons learned is also crucial, as is strengthening coordination with other actors and leveraging budgetary resources from various sources.

1 See Annex VII. Scope of work of the mid-term performance evaluation of CDLO.

1 | COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION USAID.GOV

EXECUTIVE SUMMARY

This section summarizes the methodology, findings, conclusions, and recommendations of the mid-term performance evaluation of the Community Development and Licit Opportunities (CDLO) Activity.

COMMUNITY DEVELOPMENT AND LICIT OPPORTUNITIES (CDLO)

The United States Agency for International Development (USAID) Colombia's Office of Rural Economic Development manages the CDLO Activity under the Country Development Cooperation Strategy (CDCS) Development Objective 3. Tetra Tech is the implementer of this Activity. CDLO was initiated in August 2017 with the objective of “building capacity in rural communities affected by the conflict to become reliable and effective partners of the Government and the private sector for the implementation of comprehensive rural social and economic development initiatives, including the promotion of illicit crop substitution and alternative development.” The Activity comprises four strategic, connected, and synergistic components. These components aim to 1) strengthen local communities and CBOs, 2) promote sustainable economic development, 3) contribute to providing essential goods and services, and 4) engage in multilevel dialogues for development planning. CDLO works with three types of CBOs:

economic development CBOs made up of organizations engaged in productive economic activities (hereafter referred to as “producer CBOs”); community action boards (JACs); and other CBOs, a category that comprises women’s groups, music groups, communications collectives, sports associations, and other community and civic groups. CDLO strengthens these CBOs’ capacities through hands-on exercises and specific projects related to development in their territories.

METHODOLOGY

This evaluation's main unit of analysis is CBOs, using a mixed-methods approach that combines qualitative, quantitative, and geographic analysis components.

The qualitative component sought to identify how the CDLO Activity generates processes and mechanisms that establish the results obtained by CBOs when interacting with local actors. The qualitative component sample includes 12 municipalities in nine departments. The evaluation team conducted 101 semi-structured interviews,2 ten group sessions (focus group and social mapping), and ten ethnographic tours.

MELA collected data from actors involved in CDLO activities: CBOs, CDLO national and local staff members, territorial and national level government officials, and commercial partners.

The quantitative component sought to establish the experience, achievements, and results of participation in CDLO by CBO directors and members. The instruments of the quantitative component included two surveys. The surveys characterized the CDLO Activity using information provided by CBO directors and members. The sample included a census of 309 CBOs and used a non-probabilistic sample

2 Semi-structured interviews offer the researcher considerable leeway in probing respondents, as well as maintaining the basic structure of the interview. Even if it is a guided conversation between researchers and interviewees, there is flexibility. The researcher can follow any idea or creatively take advantage of the entire interview.

USAID.GOV COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION | 2 design to gauge 1,090 members of CBO’s. The firm IPSOS collected survey data between May 1 - August 5, 2021, across 37 municipalities, 35 corridors, and nine departments.3

Geographically, the evaluation team depicted the spaces in which interactions between CBOs took place and in which relationships among them existed. The qualitative component included ten social maps, surveys of CBO directors, and secondary data sources. The approach integrated qualitative and quantitative data into maps to analyze the dynamics of these variables in the territory.

The evaluation team triangulated evidence across the different methodologies to increase the robustness of the findings. The team also used different types and sources of representative evidence, which were identified when evidence was reiterated and repeated, as a means to identify trends. The evaluation team identified 52 findings from this analysis and derived 25 conclusions and 26 recommendations from the findings. The next section summarizes the main findings and conclusions.

MAIN FINDINGS AND CONCLUSIONS

QUESTION 1: HOW HAVE THE SOCIO-ECONOMIC, INSTITUTIONAL, AND POLITICAL CONTEXT IN

CDLO'S AREAS AFFECTED THE CRITICAL ASSUMPTIONS, PERFORMANCE, AND EFFECTIVENESS OF THE

FORMULATED ACTIVITIES?

The municipalities in which CDLO operates have high poverty levels compared to non-targeted municipalities. Security indicators such as murders, extortion, and the presence of illegal armed groups also reflect worse conditions for people in CDLO municipalities than for non-CDLO municipalities.

Three out of ten CBO members declared that illicit economies are present in their veredas. The evaluation findings reflect that these contextual characteristics, especially security conditions, affected CDLO's activities in at least three ways: 1) they hindered CDLO's ability to intervene in targeted areas;

2) security conditions sometimes imposed delays or caused activities to be rescheduled; and 3) they caused CDLO to implement safety measures already used by community members in the areas (e.g., observing community curfews and visiting certain rural areas only when accompanied by community members) in an effort to protect Activity staff and partners.

CDLO activities generated clusters of licit economies when concentrated in areas with a lower presence of illicit crops and where economic, social, and infrastructure projects were implemented.

The territories in which CDLO operates overlap considerably with areas prioritized for the Government of Colombia’s development activities under Colombia’s 2016 Peace Agreement.

Communities have mostly positive expectations regarding projects such as the Territorially Focused Development Programs (PDETs), and CDLO´s interaction with local and national actors allowed for activities to be aligned to the needs and nuances of the communities; however, limitations in infrastructure and provision of basic services, as well as the low levels of institutional capacity across these territories, have affected the establishment, development, and results of CDLO activities.

3 CDLO employs a territorial development approach, which focuses on spaces that share common conditions among or within veredas – rural sub-municipal territorial divisions analogous to a small rural neighborhood – of one or more municipalities. The Activity refers to these spaces as corridors and aims to connect actions and facilitate synergies between them at local, regional, and national levels

3 | COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION USAID.GOV

CDLO has been successful in adapting the design and operation of the Activity to the characteristics of the territories. As seen during the COVID-19 pandemic, the Activity leveraged previous knowledge and experiences and adapted to changing contexts.

QUESTION 2: WHAT CAPACITY-BUILDING EFFECTS ARE SEEN IN COMMUNITY GROUPS AND

PRODUCER ORGANIZATIONS THAT BENEFITED FROM CDLO’S ACTIVITIES?

CDLO has strengthened the internal capacity of CBOs through a strategy based on the direct participation of CBOs in the design and implementation of territorial development activities. In particular, JACs have acquired the capacity to efficiently manage resources in the execution of infrastructure initiatives, improving their human, physical, and financial resource management, and promoting the formalization of contracting relationships. As a result, JACs reported having the capacity to continue executing infrastructure projects, although few have been involved in new projects.

Uncertainty exists about whether JACs have the resources to contract engineering and accounting professionals to execute future projects.

Producer CBOs reported strengthened organizational, technical, productive, and commercial capacities.

Joint work, commercial alliances with private partners, and hands-on work to design and implement strategies to respond to market demands have been crucial for capacity building.

Other CBOs have also increased their capacity to support and complement territorial development, strengthening their planning processes and diversifying their activities. In addition, CDLO built capacity among young people and promoted a generational change in CBO leadership.4

Through CDLO´s capacity building and the design and implementation of territorial development projects, the evaluation showed that the Activity promotes a culture of legality and formality and enhances the social fabric in the targeted territories.

Despite this progress, the need to enhance the capacity-building process remains – including the need ot enhance technical capacity and CBO’s ability to access financial resources. Despite these advances, there are challenges to strengthening, scaling, and achieving sustainable commercial partnerships that increase income generation.

QUESTION 3: HAS CDLO'S INTERVENTION BEEN EFFECTIVE IN CREATING A LOCAL ENVIRONMENT

CONDUCIVE TO A LICIT ECONOMY IN THE TARGETED REGIONS?

CBOs perceive a positive and improving economic environment and perceive CDLO activities as an opportunity to reduce the influence of illicit economic activities and to create clusters of licit economic activities.

CDLO has contributed to strengthening local economic development through social and productive capacity-building and through improvements in road infrastructure that have reduced travel time and

4 The evaluation team classified CBOs into three categories: i) producer organizations, which refers to economic development and commercial organizations; ii) JACs, referring to community organization boards; and iii) Other CBOs, referring to cultural and sports groups, youth associations, communication collectives, among others. When relevant, the evaluation team disaggregated the analysis into these three categories.

USAID.GOV COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION | 4 costs. CDLO´s contribution to economic development has benefited CBOs and the communities to which they belong.

Although most producer CBOs engage in traditional economic activities,5 CDLO has financed mainly non-traditional economic activities including rural tourism, gastronomy, jewelry making, and musical instrument-making, contributing to the diversification of local economies. CDLO has also stimulated income generating activities for producer CBOs, promoted the participation of young people, and contributed to reviving ancestral traditions.

CBOs perceive the relevance of CDLO as the main actor providing economic contributions, even when compared to institutional actors such as the government, the private sector, and other international development programs. CDLO has leveraged essential resources in the target territories; however, additional investments from various actors could be leveraged and coordinated. For example, access to rural goods and services, including internet and technical assistance, remains limited.

While there have been improvements in the sales and income generation of producer CBOs, challenges remain with regard to strengthening, expanding, and sustaining commercial opportunities, as only around half of CBOs surveyed reported having current commercial agreements or experiencing increased sales.

Opportunities exist to strengthen follow-up data collection on the economic outcomes of CBOs that participated in CDLO activities in order to identify areas for improvement.

Producer CBOs that have not generated income identified challenges such as a lack of access to financial resources (the CDLO intervention model did not provide this service) and a lack of equipment, technical assistance, training, and connections to forge partnerships.

CDLO has rigorously complied with USAID's environmental requirements and Colombian environmental regulations. However, the intervention model did not include specific actions related to environmental practices, which CBOs developed independently. These practices could be supported as a mechanism to strengthen added value through compliance with environmental standards that some markets may demand.

QUESTION 4: HOW EFFECTIVE HAS THE ACTIVITY´S MODEL TO PROVIDE GOODS AND SERVICES BEEN

IN REACHING ITS RESULTS?

CDLO’s model of executing public projects through JACs has been successful. Most CBOs have participated in infrastructure projects' design, execution, and follow-up. In addition, JACs have become more efficient in managing financial resources, with some even generating surpluses to finance new projects. These improvements in management have helped JACs gain internal and external legitimacy to carry out future infrastructure projects, although most have not carried out additional projects.

CDLO's goods and services provision projects have responded to the needs of CBOs and their communities. The Activity is recognized as the main actor supporting the maintenance of infrastructure.

5 According to comments on the presentation of findings and conclusions of the evaluation prepared by CDLO in November 2021, "the traditional economy involves all agricultural and agro-industrial activities with an innovative and market approach, while the non-traditional economy involves community tourism chains, creative industries and communication groups."

5 | COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION USAID.GOV

In addition, through road improvements and the provision of utilities in schools and community centers, CDLO has contributed indirectly to increasing access to essential services.

Despite CDLO´s contribution to providing goods and services, low coverage and limited access to services remains prevalent in CDLO municipalities and corridors.6 This challenge requires coordinated actions by different actors to leverage budgetary resources from other sources, which go beyond the Activity's scope.

QUESTION 5: HOW EFFECTIVE HAVE THE MULTILEVEL DIALOGUE MECHANISMS BEEN IN FOSTERING

COMMUNITY PARTICIPATION AND INCREASING THE IMPACT OF TERRITORIAL SOCIO-ECONOMIC

DEVELOPMENT DECISIONS?

The evaluation found that developing infrastructure projects with CBOs represented an initial step in building the trust required to engage actors in multilevel dialogues. Multilevel dialogue mechanisms have built and strengthened networks between CBOs, including horizontal linkages between producer CBOs working in traditional economic sectors and producer CBOs in non-traditional sectors, and between producer CBOs and other types of organizations (e.g., community communications collectives). The networks formed by CDLO-beneficiary CBOs have enabled the establishment of thriving clusters of licit economic activities where, in the past, relationships, communication, and cooperation among actors were affected by armed conflict.

CDLO has successfully built capacity, opportunities, and trust, which enables CBOs to establish better relationships with various public and private sector actors.

CDLO´s communication strategies have contributed to redefining the collective identity of stigmatized municipalities, generating a sense of belonging, pride, and self-worth. However, an opportunity remains to continue supporting and increasing these strategies' scope.

QUESTION 6: HOW EFFECTIVE HAS CDLO’S CONTRIBUTION BEEN TO THE REGIONAL DEVELOPMENT

STRATEGY DEVISED AND IMPLEMENTED BY THE GOVERNMENT OF COLOMBIA THROUGH DIFFERENT

PLANNING INITIATIVES?

CDLO has promoted the participation of CBOs in formal local engagement and planning spaces and has supported coordination and linkages with territorial and national government entities. However, in the context of CBOs working with limited infrastructure, limited provision of basic services, and low levels of institutional capacity, opportunities exist to further strengthen the process of creating connections to implement territorial development initiatives.

Various actors recognize CDLO methodologies as valuable. There is potential for these methods to be systematized and transferred to different national and local actors to share lessons and intervention models that can be scaled and replicated to other programs and public policies.

6 Some variables and indicators were calculated at corridor level.

USAID.GOV COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION | 6

EVALUATION PURPOSE

This mid-term performance evaluation´s purpose is to assess the achievements, outputs, and outcomes of the United States Agency for International Development (USAID) Colombia Community Development and Licit Opportunities Activity (CDLO) under the Activity work plan and the theory of change, establishing their relevance in the current Colombian national and local contexts.

The evaluation provides data and analyses to determine the extent to which CDLO is achieving its stated objective of “building the capacity of rural communities affected by the armed conflict to become reliable and effective partners of the Government and the private sector for the implementation of comprehensive rural social and economic development initiatives, including the promotion of illicit crop substitution and alternative development.”7

The evaluation is structured around six questions that address the influence of national and local contexts on CDLO's effectiveness, its achievements in generating community capacity, its contributions to economic development and the provision of goods and services, as well as its progress in promoting multilevel dialogue among relevant stakeholders towards development planning and its contribution to national and territorial policies and programs.

ACTIVITY DESCRIPTION AND BACKGROUND

USAID Colombia manages the CDLO Activity under Development Objective 3 of the Colombia Country Development Cooperation Strategies for 2014-2018 and 2020-2025. Tetra Tech implements the Activity. CDLO was launched in August 2017 to build capacity in rural communities affected by Colombia’s internal armed conflict, working with government and private actors to implement social and economic development initiatives. The achievement of CDLO's objectives contributes to improving relationships and interactions between the state and the community, an important development challenge in post-conflict areas. In addition, CDLO aims to increase levels of trust and legitimacy in communities to generate social capital and promote more community engagement in the public sphere and in productive activities. CDLO is implemented in 51 municipalities with a high presence of armed groups, illicit crops, and security issues.8 These municipalities are located in nine departments across six regions. Four of these regions account for 88 percent of CDLO’s activities, with their respective shares ranging from 19 to 27 percent of all activities.9 11 percent of the activities take place in the remaining two regions.10 CDLO employs a territorial development approach which focuses on spaces that share common conditions among or within veredas – rural sub-municipal territorial divisions analogous to a small rural neighborhood – of one or more municipalities.11 The Activity refers to these spaces as

7 CDLO Activity Monitoring, Evaluation, and Learning Plan, page 7 8 Source: Mission System as of September 30, 2020.

9 These regions are the Southwest (comprising the departments of Nariño and Cauca); Central I (comprising the departments of Caquetá and Putumayo), with the largest single investment share of all regions at 29.0 percent; Central II (comprising the departments of Meta and Guaviare); and North (comprising the departments of Antioquia and Córdoba).

10 These regions are the Northeast (Norte de Santander, with 2.0 percent of activities and 2.2 percent of investment), and the National level (i.e., activities implemented in all regions without regional tailoring, such as trainings on budget preparation) with

9.0 percent of activities and 12.0 percent of investment. The information is included as of September 30, 2022.

11 In Annex IV. Detailed Methodology, the territorial development model is presented. A vereda is the smallest territorial/administrative division in Colombia. Although some veredas are located in peri-urban areas, and while they may contain a hamlet as a kind of nucleus, veredas are overwhelmingly rural, with households scattered at a distance from one another within a largely rural landscape.

7 | COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION USAID.GOV corridors and aims to connect actions and facilitate synergies between them at local, regional, and national levels.

CDLO implemented 129 activities between its implementation and September 30, 2020.12 The main components are as follows: 41 percent relate to socioeconomic development; 34 percent to infrastructure and public service improvement; 12 percent to creating multilevel dialogue mechanisms;

12 percent to strengthening community organizations; and one percent to cross-cutting activities. The economic development component and provision of goods component each account for 41 percent of a total USAID investment of COP 68,649 million.13

The direct beneficiaries of CDLO are 425 community-based organizations (CBOs) in the intervention territories.14 Three types of CBOs participated in the Activity: producer organizations, community action boards (JACs), and a range of civic and cultural groups, including women’s groups, youth groups, sports associations, and communication collectives, which the evaluation refers to as Other CBOs.

Of these CBOs, 45.4 percent are linked to the economic development component (producer CBOs);

33.6 percent to the provision of goods and services (JACs); 14.2 percent to strengthening organizations;

and the remaining 6.8 percent to the creation of multilevel dialogue mechanisms.

The Activity established four strategic, articulated objectives, which aim to complement each other and create synergies that broaden the impact of the Activity.15 Each objective is directly connected to a CDLO component, presented below.

CDLO COMPONENT 1. Strengthening local communities and CBOs (referred to herein as organizational capacity building) that have been affected by the conflict, as well as strengthening government actors acquiring capacities and empowering them to become reliable partners for the implementation of local development activities. CDLO contributes to capacity building in business management, customer service, marketing and communication strategies, information and communication technology, and management of commercial events such as trade fairs. CDLO tailors its capacity building activities to the type of CBO and the CDLO activity in which CBOs engage. Under the CDLO model, CBOs strengthen their capacities mainly through hands-on exercises, engaging in designing and implementing productive and infrastructure-related activities with guidance from CDLO.16 Organizational capacity building is understood as a cross-cutting axis that enables other objectives and components of CDLO. 80 percent of CDLO CBOs are involved in the Activity’s other three components (i.e., economic, infrastructure, and communications) to implement the organizational capacity-building component.17 Of the surveyed CBOs, 42.7 percent are linked to the economic component, with 76.5 percent of these being agricultural producers. In turn, 32.2 percent of the CBOs are related to infrastructure activities, with 95.6 percent of these being JACs. The remaining 20.1 percent of CBOs participate in the capacity strengthening and multilevel dialogue components. Among

12 This evaluation was carried out based on this group of activities, which are classified as closed, in closure, and in execution.

This information was taken from Mission Activity, October 1, 2021.

13 The remaining 17.6 percent of investment resources correspond to 9.4 percent for the creation of multilevel dialogue mechanisms; 8.1 percent for strengthening of community organizations; and 0.1 percent for cross-cutting activities.

14 Source: Mission System as of September 30, 2020.

15 Document Section C of the contract update, page 1.

16 Economic and infrastructure component activities represent 82.5 percent of the resources invested in CDLO.

17 There are three types of CBOs: producer organizations, JACs, and Other CBOs (which include organizations such as communications collectives, youth groups, sports groups, and women's groups).

USAID.GOV COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION | 8 other CBOs, 31.9 percent participate in the economic component and 8.5 percent in the infrastructure component. The other CBOs were involved in organizational capacity building and multilevel dialogue activities.

The implementation of CDLO activities allows producer CBOs to manage elements of the creation or consolidation of their businesses, such as problem-solving to promote value chains, marketing, or the representation of their members.18 For JACs, the implementation provides hands-on experience in the provision of goods and services, as well as in exercising effective oversight.

Capacity building is a cross-cutting component of CDLO's model. CDLO's premise is that “a strengthened CBO will be better at providing services to its members and communities. This promotes cooperation among citizens and results in [a greater sense of belonging and connection to] the territory and greater integration with the public and private sectors.”19 In other words, the CDLO model considers a strengthened CBO to be a more capable and effective partner to the public and private sectors.

CDLO COMPONENT 2. Linking organized communities to provide essential goods and services and improve and maintain infrastructure in conflict-affected areas (referred to herein as the provision of goods and services). Under this component, CDLO facilitates links between CBOs to develop social and productive infrastructure projects and provide goods and services. Through their work, CBOs contribute to the development and viability of the activities. Through hands-on exercises, the CBOs involved in this component, almost all of which are JACs, participate in the definition, design, implementation, and oversight of infrastructure works. Through this experiential learning (learning-by-doing) process, the JACs acquire the necessary technical, administrative, and organizational capacities to actively manage new agreements for small infrastructure projects with territorial and national entities and the private sector.20 In addition, the infrastructure component of CDLO helps build initial trust in communities because it allows the Activity and the JACs to demonstrate a tangible achievement to the targeted communities.

CDLO COMPONENT 3. Establishing links between producer organizations, local associations, productive partnerships, and local and regional businesses through the design and implementation of rural development activities (in traditional, non-traditional, and infrastructure sectors of the economy).

The rural development activities align with the productive vocation of the territory (that is appropriate to the local context in productive and cultural terms) to promote sustainable economic development in conflict-affected areas (referred to herein as economic development). This component also promotes and strengthens partnerships and supply chains linking entrepreneurs, producers, and local and national markets. Thus, it seeks to help build the conditions for inclusive rural economic growth and development opportunities to solidify a lasting and sustainable peace. CBOs that participate in this component strengthen their capacities through hands-on exercises as they develop their productive activities.

18 Comments on the presentation on findings and conclusions of the evaluation prepared by the implementing partner. The representation of its members refers to matters related to the business of the CBOs, as well as in different spaces of participation and decision with the public and private sectors.

19 CDLO. Monitoring and evaluation guide. Page 42.

20 Ibid.

9 | COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION USAID.GOV

CDLOs’ activities seek to respond to rural communities’ expectations and promote licit and sustainable economic growth through strengthening and innovating traditional and non-traditional economies and encouraging the development of licit economic clusters in the corridors. Boosting local economies entails providing technical and organizational services and infrastructure to support the development of productive activities directly, which also depends on linking producer groups to markets through private sector partners. Producer CBOs are the leading actors of CDLOs’ economic component, which aims to contribute to capacity building, improve productive activities, and the establish commercial agreements.

CDLO COMPONENT 4. Community engagement in multilevel dialogues for development planning in conflict-affected areas (referred to herein as the creation of multilevel dialogue mechanisms).

Strengthening communication mechanisms between the state and local communities establishes community organizations as legitimate representatives who can develop effective dialogue channels and partnerships with other actors. This enables communities to engage in decision-making and management processes related to territorial development.21 These dialogues allow organizations to develop relationships and influence government and private sector actors. Engaging in dialogues with other actors allows JACs to adopt procurement models for small public and private infrastructure projects, leveraging funds from foundations, oil companies, and other organizations. With the guidance of CDLO and an experiential learning approach, the JACs generate the capacity to develop infrastructure projects.

CDLO also facilitates capacity building for communications collectives to design and disseminate messages and promote the productive activities and the provision of goods and services developed by the CBOs in other CDLO components. CDLO does not intend for communications collectives to lead multilevel dialogue processes.22 Instead, these collectives promote communication for the distribution, promotion, design, and marketing of local processes and brands that contribute to development.

Multilevel dialogue is the transversal component of CDLO, which seeks to connect and consolidate its four components. Multilevel dialogue creates synergies between the different elements of CDLO (objectives, components, activities, and actors) and promotes the sustainability of the Activity. CDLO’s purpose is to strengthen the capacities of CBOs through hands-on exercises, participation in decision-making on infrastructure site selection and design of activities, as well as in the implementation and community oversight of activities. It ultimately aims to achieve (without this being an expected result of CDLO) community management and participation in decisions that concern territorial development.

21 Comments on the presentation on findings and conclusions of the evaluation prepared by the implementing partner.

22 Ibid

USAID.GOV COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION | 10

EVALUATION QUESTIONS AND METHODOLOGY

This evaluation answers the six questions outlined in the Scope of Work document. Each of the questions considers the context and scope.23 This section presents each evaluation question (in red) and the information, data, and other inputs used to answer each question.24 The evaluation team defined an evaluation methodology and an evaluation matrix in line with the objectives and conditions of CDLO implementation. The evaluation team also established the evaluation categories of analysis for each question.

EVALUATION QUESTIONS

Q1. HOW HAVE THE SOCIO-ECONOMIC, INSTITUTIONAL, AND POLITICAL CONTEXTS IN CDLO'S

AREAS AFFECTED THE CRITICAL ASSUMPTIONS, PERFORMANCE, AND EFFECTIVENESS OF THE

FORMULATED ACTIVITIES?

CRITICAL ASSUMPTIONS. The activities carried out by CDLO are based on five critical assumptions:

• CDLO activities should employ an approach that meets the needs of the geographic corridors, incorporating the differences across populations and infrastructure and enhancing rural investment.

• The success and sustainability of CDLO activities largely depend on the security conditions that the Government of Colombia can provide in the corridors.

• Infrastructure limitations and low levels of investment in the corridors should be considered within CDLO activities.

• Creating synergies and collaborative contributions among actors to compensate for the territories’ limited resources and rural infrastructure needs is important; and

• CDLO prioritizes local corridors according to their strategic importance and security levels. The evaluation assesses CDLO´s performance and effectiveness, considering how the corridors' context affects CDLO implementation to answer this evaluation question.25

CONTEXT VARIABLES. The evaluation approaches the analysis of the context using the following four categories of variables: 1) social coexistence and conflict related to public security; 2) illicit rural economies related to illicit crops and drugs, illegal mining, and other illicit economic activities; 3) political and institutional context, addressing institutional services within the framework of the Peace Agreement;

and finally, 4) the municipal socioeconomic context, which includes multidimensional poverty levels, the coverage of social services, employment, and income growth variables, and the effects of COVID-19.

23 See Annex VII. Evaluation Scope of Work, Evaluation Questions section.

24 In Annex IV. Detailed Methodology, the design matrix is presented and details on how the evaluation team answered each question are presented.

25 The evaluation team defined effectiveness as the fulfillment of the Activity’s objectives.

11 | COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION USAID.GOV

Q2. WHAT CAPACITY-BUILDING EFFECTS ARE SEEN IN COMMUNITY GROUPS AND PRODUCER

ORGANIZATIONS THAT BENEFITED FROM CDLO’S ACTIVITIES?

CAPACITY STRENGTHENING VARIABLES. The evaluation team defined two categories of analysis to structure the findings: 1) organizational capacity (e.g., governance, administrative, planning, human resources, and financial management) and 2) social and community capacity (e.g., community ties, conflict management, trust, shared values, and visions for territorial development). The evaluation team also identified CDLO strategies and capacity-building tools (e.g., selecting participants, methodologies, scope, and achievements).

Q3. HAS CDLO'S INTERVENTION BEEN EFFECTIVE IN CREATING A LOCAL ENVIRONMENT CONDUCIVE

TO A LICIT ECONOMY IN THE TARGETED REGIONS?

ECONOMIC DEVELOPMENT VARIABLES. The evaluation team applied three categories of analysis for economic development: 1) the harnessing of economic opportunities related to the diversification of productive activities, innovation processes, and the assessment of local assets; 2) boosting the local economy, which is associated with linking markets, capacity support services (technical, financial, and commercial), and developing productive and social infrastructure to support the licit economy; and 3) the economic, financial, social, and environmental sustainability of activities.

Q4. HOW EFFECTIVE HAS THE ACTIVITY´S MODEL TO PROVIDE GOODS AND SERVICES BEEN IN

REACHING ITS RESULTS?

INFRASTRUCTURE VARIABLES. The evaluation team established two categories of analysis for infrastructure projects: 1) provision of public social goods and services (i.e., access, coverage, functionality, and quality of the services provided); and 2) community involvement in the provision of goods and services (experiences of communities in the provision of goods and services and related coordination with the institutional framework).26

Q5. HOW EFFECTIVE HAVE THE MULTILEVEL DIALOGUE MECHANISMS BEEN IN FOSTERING

COMMUNITY PARTICIPATION AND INCREASING THE IMPACT ON TERRITORIAL SOCIO-ECONOMIC

DEVELOPMENT DECISIONS?

MULTILEVEL DIALOGUE VARIABLES. The evaluation team established two categories of analysis: 1) multilevel dialogue mechanisms between CBOs, and 2) multilevel dialogue mechanisms with the public and private sectors. These categories address variables related to networking, community relations with interest groups, communication for development, citizen participation, and coordination and connection processes.

Q6. HOW EFFECTIVE HAS CDLO’S CONTRIBUTION BEEN TO THE REGIONAL DEVELOPMENT STRATEGY

DEVISED AND IMPLEMENTED BY THE GOVERNMENT OF COLOMBIA THROUGH DIFFERENT PLANNING

INITIATIVES?

THE CDLO MODEL assumes that in the local context and through participation or involvement in existing public policy planning processes, CBOs and institutional and private sector partners can create

26 The institutional framework refers to local level institutions in charge of developing public works or interventions, such as mayors’ offices and municipal secretariats.

USAID.GOV COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION | 12 synergies to enhance the scope of the activities developed. Question six assesses how the different community actors linked to CDLO have contributed to the development of this model in the territories. It is worth noting that this development contribution is not an expected purpose for all the activities carried out by CDLO, nor does CDLO intend to influence public policy directly. Instead, the evaluation understands CDLO's contribution to the territorial development strategy as enabling the Activity’s CBO beneficiaries to participate in the planning processes of existing public policy.

TERRITORIAL CONTRIBUTION VARIABLES. The evaluation team approached this question using three categories of analysis: 1) the Activity´s contribution to the design of public policy (diagnoses and formulation of territorial development plans, programs, or works at the national, departmental, and municipal levels); 2) contribution to implementation through complementary investments or co-financing from different levels of government; and 3) the revision or reframing of public policy through CDLO's conceptual, methodological, operational, and evaluative contributions based on its intervention methodologies.

EVALUATION METHODOLOGY

The evaluation is designed to address the results of CDLO's objectives related to its direct beneficiaries, the CBOs carrying out social and economic activities in the geographic corridors. The geographic corridors are made up mainly of intra-municipal groupings of veredas. The evaluation includes 129 completed or in-progress activities as of September 30, 2020.27

The methodological approach of the evaluation was designed as a mixed-methods design, combining three components: qualitative, quantitative, and geographic. This design allows the identification, assessment, and in-depth understanding of relationships between variables, using a variety of primary and secondary data collection instruments and analytical methods. The methodological approach to answering the evaluation questions was structured based on two principal elements: 1) aspects common to all evaluation questions (i.e., those systematically used to address each of the evaluation questions, such as the calculation and analysis of indicators from the surveys applied and triangulation with qualitative information from interviews and group activities), and 2) specific elements applied to some of the questions according to their characteristics and the purpose of analysis (e.g., the development of regression models to find variables that were associated with the strengthening of economic activities).28

QUALITATIVE COMPONENT

The qualitative methods used in the evaluation are intended to provide detailed data on how the CDLO intervention generates processes and mechanisms that determine the CBOs’ results when interacting with local actors in the geographic corridors. The analysis uses a value mapping framework focused on the following: captured value, lost value, destroyed value, new opportunities for value generation, and territorial competitiveness. “Captured value” refers to positive benefits generated by CDLO for each CBO or in its context (i.e., within a corridor).29 “Lost value” refers to inefficiencies in CDLO's

27 Of these 129 activities, 53 are economic, 44 are related to infrastructure, 16 with multilevel dialogue, 15 with CBO strengthening, and one is cross-cutting.

28 Annex IV. Detailed Methodology includes the evaluation matrix, which specifies how the team approached each of the questions and the informants for each dimension of the analysis. Annex V. Instruments includes the data collection tools and Annex VI. Data Sources includes the quantitative sample and data sources.

29 See Annex IV. Detailed Methodology, Qualitative Data Analysis section.

13 | COMMUNITY DEVELOPMENT FOR LICIT OPPORTUNITIES MID-TERM PERFORMANCE EVALUATION USAID.GOV interactions with CBOs or missed opportunities that do not allow CBOs to improve the situation of beneficiaries. Lost value is associated with perceptions of CDLO’s interactions with CBOs that could have been stronger and may be related to bottlenecks including lack of capacity, low follow-up on activities, or lack of a connection among actors. “Destroyed value” refers to negative assessments of the organizations’ interactions with CDLO and is associated with reputational risks for CDLO and/or actions that generate conflicts between organizations and/or individuals. “New opportunities for value generation” refers to CBO proposals for process optimization, improvement of individual living conditions, and/or new forms of action for the organizations. Finally, “territorial competitiveness” is understood as a shared perspective regarding the territory based on elements such as endogenous development potential, valuation of the territory’s resources, relationships among the actors involved, and innovation.

SAMPLE

The evaluation team applied six criteria to select the 12 municipalities across nine departments, which made up the qualitative sample for the evaluation: geographic coverage, level of execution, diversity of activities, access to corridors, CDLO funding, and private sector investment.30

The evaluation team conducted 101 semi-structured interviews, ten group sessions (focus group and social mapping), and ten ethnographic tours between February 8 and April 19, 2021.31 The team collected information from the actors involved in CDLO activities: CBOs, the CDLO team, the territorial and national government implementers, and commercial partners.32 The topics addressed in the different interviews, sessions, and ethnographic tours were defined according to the characteristics of each question, the conceptual and analytical criteria established in the evaluation design matrix, and the relevance of the stakeholders for different aspects.

ANALYSIS

The evaluation team analyzed the qualitative information at two levels: 1) coding of the transcripts using…