Attachment J.28 ADAM & MIDAS POST IMPLEMENTATION EVALUATION REPORT.pdf

March 2014 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by the Evaluation and Analysis for Learning (EVAL) project, and its team members Management Systems International (MSI) and Centro Nacional de Consultoría (CNC). The report was written by Oscar Huertas Diaz, Director of the Evaluation, and Keri Culver, Lead Evaluation Specialist for EVAL.

Post-implementation evaluation report:

Evaluation of the More Investment in Sustainable Alternative Development and Areas for Municipal Alternative Development programs

FINAL REPORT:

POST-IMPLEMENTATION EVALUATION

Post-implementation evaluation of the programs More Investment in Sustainable Alternative Development (MIDAS) and Areas for Municipal-Level Alternative Development (ADAM)

Management Systems International Corporate Offices 600 Water Street, SW Washington, DC 20024, USA Tel: + 1 202 484 7170 / Fax: +1 202 488 0754

Contracted under AID-514-C-13-00003

Evaluation and Analysis for Learning (EVAL)

DISCLAIMER

The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government.

FINAL REPORT:

POST-IMPLEMENTATION EVALUATION OF THE PROGRAMS MORE

INVESTMENT IN SUSTAINABLE ALTERNATIVE DEVELOPMENT

(MIDAS) AND AREAS FOR MUNICIPAL-LEVEL ALTERNATIVE

DEVELOPMENT (ADAM)

CONTENTS

Acronyms ....................................................................................................................................... v Executive Summary ..................................................................................................................... vi Introduction

Evaluation Purpose Evaluation Questions

Project Background Evaluation Design

Evaluation Methods Sampling Limitations and Gaps

Findings, Conclusions & Recommendations

1. Key Theme: Associativity

2. Key Theme: Targeting and planning for agricultural projects

3. Key Theme: Marketing, competitiveness and food security with vulnerable groups

4. Key Theme: From vulnerability to resilience: the tipping point out of subsistence

5. Key Theme: Coordination with municipalities and the state

6. Key Theme: Capacity building

7. Key Theme: Engaging the whole community

8. Key Theme: Differentiated approaches for vulnerable groups

9. Key Theme: Access to financial services and formalization

10. Key Theme: Building a culture of legality

11. Key Theme: Sustainability and replication

12. Key Theme: Managing implementation with indicators

13. Key Theme: Targeting beneficiaries with different needs

USERS’ Guide: Linking ADAM/MIDAS Lessons to Post-Conflict Programming Beginning to connect the dots to the CDCS and the peace process Examples to guide further analysis

Working with Producer Associations: a two-for-one proposition

Transitioning from a life based on coca to a culture of legality Kick-starting Integrated Rural Reform through Peace Pollinators Scaling up alliances as part of RRI Linking productive projects to local government

Further applications of ADAM/MIDAS learning

Table of Figures Figure 1. Evaluation case studies Figure 2. Summary of the site visit evaluation sample Figure 3. Summary of the quantitative survey sample Figure 4: New crops required advance soil and climate study Figure 5: Associations’ ability to market critical in finding alternatives to coca Figure 6: Using test laboratories to train vulnerable women increased employability Figure 7. Recommendations by key theme Figure 8: Evaluation themes relevant to specific points in the Colombian peace negotiations Figure 9. Implementation of peace likely to follow negotiation points in Havana Figure 10. Successful Association work accompanies them to market Figure 11. Cacao production has helped foster a culture of legality in Tumaco v

ACRONYMS

Note on the use of conventions: This report uses the standard convention of italicizing words in Spanish, (e.g., veredas or campesinos), with the exception of proper nouns (e.g., Banca de Oportunidades.) A second convention involves the capitalization of acronyms that have become words, such as the names of organizations, are spelled with only the initial letter capitalized (e.g., Asocati, Apropesca, etc.) and these are not included in the Acronyms list.

ADAM Areas for Municipal-level Alternative Development (ADAM for its Spanish initials)

CDCS Country Development Cooperation Strategy

CONPES Consejo Nacional de Política Económica y Social

CS Case study – report method used to illustrate models, best practices, and lessons learned

DIAN Dirección Nacional de Impuestos y Aduanas

DPS Departamento para la Prosperidad Social (successor to Acción Social)

ECA Escuelas de Capacitación para Agricultores (farmer training schools)

EVAL Evaluation and Analysis for Learning

GE Gente Estratégica

GOC Government of Colombia

HA Hectares

KT Key Themes – ordering structure of the report’s findings, conclusions and recommendations

MIDAS More Investment in Sustainable Alternative Development (MIDAS for its Spanish initials)

PyMEs Pequeñas y Medianas Empresas (Small and Medium Enterprises)

SENA Servicio Nacional de Aprendizaje

SME Small and Medium Enterprises

SOW Statement of Work

TA Technical assistance

TEP Productive Ethnic Territories (TEP for its initials in Spanish)

UMATA Unidad Municipal de Asistencia Técnica Agropecuaria

USAID U.S. Agency for International Development

USG U.S. Government vi

EXECUTIVE SUMMARY

Evaluation Purpose and Evaluation Questions The purpose of the evaluation, coming as it does over three years post-implementation, is not to judge the success or difficulties of the programs under study. Rather, this review of lessons learned and detailed case examples are designed to inform post-conflict Mission programming.

The evaluation questions examine best practices and lessons learned and extract useful methods, processes, capacities, alliances and activities undertaken in four key program components. The evaluation is designed to demonstrate degrees of success and sustainability, to support Country Development Cooperation Strategy (CDCS) implementation and to provide the ‘why’ and ‘how’ to program designers and activity managers.

Project Background The United States Agency for International Development (USAID) created two major programs in Colombia to improve conditions for rural citizens through productive projects, community participation, public policy development and strengthening municipal governments. Many Colombian organizations worked in partnership to extend the influence of programming, ensure relevance, and work toward sustainability. ADAM and MIDAS offered alternative development options in complex, multi-faceted programming designed to affect social and economic behaviors at local, municipal and national levels. They operated in environments where illicit crops, displacement, violence and environmental effects were always threats.

ADAM (US$189,000,000) focused on productive agricultural projects, public works, municipal strengthening and civil society collaboration. These tended to be deep interventions in a given community, with an integrated model, and were implemented in 75 municipalities. MIDAS (US$180,000,000) provided technical assistance and support to micro-, small- and medium-enterprises, productive forestry, conservation, and agribusiness, in more than 500 municipalities.

Its policy component supported reforms for development at all levels of the country.

Evaluation Design, Methods and Limitations The evaluation used a mixed methods design with triangulation of sources. Quantitative methods included infrastructure observations and a business owner survey. Qualitative methods began with a review of MIDAS and ADAM documents. The team conducted interviews and focus groups with USAID, implementers, agribusiness producers and associations, second-level organizations, small-and medium-sized enterprises, councils and community members, and parastatal and Government of Colombia (GOC) respondents. Fieldwork reached 249 people from thirteen departments and 57 municipalities. Field teams traced lessons, practices and cases to understand their implementation.

The evaluation was carried out over three years after programs’ end. As such, the sample is limited to respondents with up-to-date contact data, or who could be found through a field contact or “snowball” sample (please see the Design chapter for further information on this method.) The findings, therefore, are not representative across the programs. However, the purpose was not representativeness or judgment of the program; rather, it was to extract lessons and practices to inform programs in light of the peace process. For this purpose, the methods were ideal.

Findings, Conclusions and Recommendations The full set of findings, conclusions and recommendations can be found in Annex I in tabular form. In this summary, the evaluation presents its overarching conclusions and recommendations.

vii

Sustainability. The evaluation found many sustained organizations, infrastructure works and productive activities, in all departments. Communities and projects were highly motivated, even in the face of adversity, to succeed and expand. Newer associations, especially with unfamiliar crops, needed more support, and rates of failure were higher. Recommendations include:

o Leverage positive experiences with current and past community-level projects, companies and public works by having them share their experiences in peer-to-peer learning.

o In exit strategies, link producers to networks – peers, as above, but also market actors, government and university technical assistance sources, and financial resources.

Associativity. ADAM and MIDAS realized great gains in building associativity among some of their productive and public works activities, helping communities begin to reconstruct social fabric damaged by conflict, illicit livelihoods, poverty and the absence of the state. The common goals, training, and support were essential, yet took time to build. Notably these successes were also replicated across communities – when projects involved some combination of campesinos, indigenous groups, and Afro-Colombians. Recommendations include:

o Use genuine participatory methods in bottom-up decision-making on productive and public works projects. Build in time for the process, particularly with new associations or groups.

o Diagnose local needs and priorities around associativity, in an inclusive, intergenerational process. This is often promised, but in the rush of implementation it can be compromised.

o Train and monitor implementing staff on inclusivity and respect for self-determination.

Culture of legality. Families and communities embraced licit opportunities to make a living and avoid the social disintegration, risks to life and livelihood, and shadow markets of illicit crops.

The evaluation recommends:

o Focus on what beneficiaries need in order to think long-term: credit access, land tenure, education and capacity building options.

o Foster this culture with the same conditions that supported associativity.

Scaling up. Established enterprises and associations provided fertile ground for productive activities that change mentalities and scope from subsistence, household levels to efficiency, marketing and a regional economy. Given the ambition and global scope of the proposed peace process, working with the GOC at all levels will be essential. Recommendations include:

o Build with experienced associations and companies using the bottom-up, participatory approach to take best advantage of their proven interest and ability to produce.

o Scale projects sufficiently for production and livelihoods that are greater than subsistence level. This goal should supersede attaining output indicators.

o Ensure municipal participation. The peace process should empower municipalities, and their buy-in is a precondition for sustainability and state credibility.

Monitoring and Evaluation (M&E). The output-level indicators by which ADAM and MIDAS are known – families supported, hectares planted, jobs created – were flawed both for telling the story of the programs and for monitoring their progress to improve implementation.

o Create modern M&E systems for complex environments - adaptive, shorter-cycle, context-and conflict-sensitive, and built around shared stakeholder goals. Avoid privileging output-level results that distort implementation by affecting implementer incentives.

o Work to instill an evaluation culture that permits experimentation in the Mission.

o Manage knowledge intelligently and enlist successful partners to share with others.

INTRODUCTION

Evaluation Purpose USAID/Colombia commissioned this external, post-implementation evaluation of aspects of its Programs entitled “MIDAS (Más Inversión para el Desarrollo Alternativo Sostenible)” and “ADAM (Áreas de Desarrollo Alternativo Municipal)” to inform pending program designs. This report will help the Mission better understand what worked, did not work, and what has been sustained, by documenting lessons and case studies from a set of ADAM and MIDAS activities.

In that it does not judge success or failure of the projects, this evaluation is atypical. More than three years post-implementation, the activities and their participants have been altered by external events, to the point that judgment would not be possible. Rather, the evaluation is designed to capture and disseminate lessons learned on key activities, sectors, and cross-cutting themes, by examining the activities that remain active today, or that can provide lessons learned since their termination about implementation and sustainability. The evaluation findings will be incorporated into program design in alignment with the new Mission CDCS. In this way, the evaluation will be used to support evidence-based activity design and implementation.

The results and findings of the assessment will be used by (i) USAID/Colombia Mission Front Office, Technical Offices, and Program Office and (ii) implementing partners. These results can also provide important inputs to the Government of Colombia (GOC) for replication, sustainability, and in implementing the peace process.

Evaluation Questions The evaluation examined a set of activities within two guiding principles:

1. Identify best practices and lessons learned that can be taken from ADAM and MIDAS to inform USAID/Colombia’s CDCS implementation, including new program development and new activity design.

2. Document and describe cases (e.g., success stories) where methodologies, processes, capacities, public-private partnerships and infrastructure set in motion by the projects have been successful and sustainable.

The activities under study in the evaluation are examined with respect to these two guiding principles. The activity sets are as follows: (full description of the areas of inquiry can be found in Annex XI in the Evaluation Statement of Work.)

1. Agricultural activities (ADAM outcome 2) and productive activities (MIDAS outcome 1)

2. Community participation (ADAM outcome 3)

3. Municipal strengthening (ADAM outcome 6)

4. Alliances (MIDAS outcome 3)

The evaluation team sought to unpack these activity sets with attention to key factors related to each thematic area. These key factors are listed in full in Annex IV Getting To Answers table.

The report continues with background on the ADAM and MIDAS programs, evaluation design and limitations, and the main body of the report – Findings, Conclusions and Recommendations, which include “meta” conclusions and recommendations that unite findings from multiple key themes. An added section discusses these findings in light of a likely peace accord, and how to make best use of lessons learned in new programming plans. Readers are encouraged to review the evaluation case studies in Annex V as well as the full set of findings in Annex I.

A summary of these case studies is offered in Figure 1, below. The case studies offer insights into the opinions, perspectives and dynamics of the ADAM and MIDAS activities at “ground level.” The cases generally show both positive effects and lessons for future programming. The cases demonstrate the underlying “why” and “how” of the various models used by implementers and operators to work toward their desired results.

Figure 1. Evaluation case studies

Beneficiary Location Sector Description of the case

1. Aproaca - ADAM Antioquia Cacao

This association benefited new cacao production with commercialization, plant material, TA and support for project implementation. As a result of ADAM participation, Aproaca became providers of TA and transporters. They earned an Alianza Productiva project with the GOC. Casa Luker buys their production. The group has increased output and quality to meet buyers’ needs. There is new citizen empowerment and oversight in the organization.

2. Aprocasur -

ADAM

Sur de Bolívar

Cacao, coffee

Organized as a second-level association under ADAM, Aprocasur has grown to the lead National Cocoa Network with 15,000 families, 28,000 cacao hectares and 40 organizations in 17 departments. Aprocasur provides TA, institutional strengthening and microcredit to its members.

They built the ADAM program through a bottom-up approach on which the organization insisted, with the intent of empowering small farmers. Successful marketing of much of the production.

3. Aprolim - ADAM Nariño Dairy

This study of an Indigenous group includes the story of one woman's empowerment. As the project began, the community found itself without a Legal Representative and with little much faith in associativity. The woman put herself forward for leadership, oversaw community training, and worked to convince municipal and departmental governments until they helped buy a chilling machine. Challenges remain, including some beneficiary unwillingness to change their milking practices to newer, more modern methods.

4. Apropesca - ADAM Cauca Fish

This 30-year-old association produces fish in Cauca, and involves Colombians from five different, and formerly competing, indigenous communities, plus a campesino group. With ADAM´s help, they created space for dialogue and conflict resolution over land issues, and increased income.

Some negative environmental impacts resulted from a lack of technical assistance, and a local mayor is attempting to take the land on which a viscera processing plant is built, for lack of written title.

5. Asocati -

MIDAS

Norte de Santander Cacao

Asocati has become a national-level actor in the promotion of sustainable cacao as an alternative to illicit crops, particularly in conflict areas, and expanding the culture of legality. These changes occurred during their MIDAS activities. Asocati´s size and influence impact the degree of industrialization and the coverage of their association and its benefits. Their chief obstacle is infrastructure to expand coverage and opportunity. Knowledge exchange has a prime best practice that this case demonstrates well.

6. Asocoprolyda

- ADAM Nariño Dairy

In this association campesino and indigenous dairy producers come together with strong evidence of gains in associativity. There are also ample challenges: disagreements and project desertion.

Quality and efficiency gains, environmental improvements and certifications supported those who stayed. The community also challenged the wisdom of the TA provided, and had to face challenges with municipal participation, co-optation, and funding. The case provides useful lessons for overcoming differences through negotiation.

7. Asogpados -

MIDAS

Norte de Santander Palm

In this case a palm promoter became a MIDAS operator after once having received USAID funding as a beneficiary. With MIDAS, the association promoted producer relationships and provided alternatives for campesinos who formerly grew illicit crops. One strength was supporting smaller associations with administrative and management training, organizational development, TA and market links. They also supplied low-cost inputs through their own supply. They are working now to build a processing plant and roads.

8. Asopalmira – ADAM & MIDAS Nariño Cacao

This community was deeply damaged by a natural disaster in 2009. MIDAS arrived to support the community and worked with the municipality to support a rapid and comprehensive change to cacao production. Families created new life projects. An ADAM school construction was taking place at the same time, with significant community involvement. Respondents reported that the efforts had changed their mentality and made them more entrepreneurial, focusing on market-driven production and marketing.

9. Asopez – ADAM Putumayo Fish

This project was situated in an area with ongoing conflicts over illicit crops and an oil pipeline.

ADAM strengthened an existing fish farm/processor with technology and TA, systematizing their administrative and management of the livestock. The program featured a strong focus on crossing generations - youth participation, studying with SENA, doing study tours. Infrastructure and other weaknesses affect successful marketing.

10. Aspalbe - MIDAS Cesar Palm

In an area that once suffered from some of the highest statistics for violence and victims of violence, MIDAS supported the oil palm company Palmariguani. The company guaranteed small producers’ bank loans, purchased their production, and offered transportation. Beneficiaries praised the interactive, practical training and the childcare provided to facilitate mothers’ attendance. This was a win-win for the processor as well, because it resulted in more hectares under production, higher yield and better quality. Associativity provided beneficiaries with some protection from armed groups as well.

11. Compañía de Empaques, S.A. - MIDAS

Antioquia, Nariño, Cauca

Fique

MIDAS supported this commercial firm to train producers in twelve associations across three departments. The company set up collection points, increased plantings and yield, and provided TA in processing, quality and transport. They removed intermediaries from the value chain, increasing value for themselves and for producers. The company asked SENA to train widely to meet excess demand with the currently unprocessed supply. The company noted the long process needed to bring small producers to the necessary levels of production and quality, and in basic business topics.

12.

Condimentos Putumayo –

ADAM & MIDAS

Putumayo Spices

Production of peppers was, for many small producers in this project, their first work after voluntarily eradicating coca. It has also recuperated deforested areas. In the area, this project is nearly uniquely successful; many in the region have failed. Thought difficult to start, because the product was new, the growers have made progress and markets are slowly coming on line.

Plantation size per family is not large enough to affect family economy. The physical plant is underused at about 5% of capacity. Respondents appreciate being back in the licit economy, including positive changes for families and communities.

13. Fedar - ADAM Cauca Youth

ADAM created a project with FEDAR which brought together youth from municipalities hit by conflict. They were trained in their territories and natural resources, thinking critically, service projects, and interpersonal and political communication. Participants reported greater ability to resist recruitment by armed groups, and social fabric across generations. Students submitted an education proposal to the local government; at the same time they built links with national and international groups. The method was shared with the National Coffee Growers’ Federation, who used the same tools with young coffee growers.

14. Frepac - ADAM Cauca Dairy

In this project both the association and the crop were new for beneficiaries, bringing technical, administrative and interpersonal challenges. TA included neither an adequate soil study nor respect for local climate knowledge. Inferior seedlings resulted in the loss of an entire harvest.

The second-level association appears to have done a poor job in TA and follow-up, particularly on credit. Some families are indebted to the present day. However, there were some positive elements in this challenging case. The association involved afro-Colombians and indigenous communities, a new collaboration for the participants. Respondents felt they learned a great deal about farming techniques from the training.

15. Gente Estrategica -

MIDAS

Atlántico SMEs

With MIDAS support Gente Estrategica, an outsourcing and human resources management firm, created more than 3,000 new jobs in 18 months, 2,700 in the lowest social strata. The company grew from 10 to 63 branches country-wide. Their work includes special attention to Afro- Colombians in vulnerable communities, and capacity building in basic work skills, self-esteem and other themes. The company now also implements for cooperation agencies and international foundations. A strong business vision and plan helped target new employees for industries in need, providing well-trained and adapted staff.

16. Greystar - MIDAS Santander

Small-scale agri-culture

Mining company GreyStar worked through its foundation and MIDAS to conduct training in small-scale agricultural production with spouses of their employees, primarily women. The women learned about entrepreneurship, good practices on small farms, and marketing their produce. The project was welcome at the time but became more important after MIDAS left, when the region was given an environmentally protected status, and all mining stopped. Working with the municipality, the women now sell their surplus in a central market. Their produce provides dietary diversity and likelier food security.

17. Red Cantoyaco -

ADAM

Putumayo Radio

ADAM supported radio station network in Puerto Caicedo featuring community programs: shows made by and about women, indigenous communities, afro-Colombian groups, and youth. A popular radionovela was created with protagonists facing local problems, such as the dangers of illicit crops and resisting armed actors in the community. The municipality invited listener comments to their show. The station revived a local musical style, and now plays the music of several bands that were created for a biennial concert. ADAM supported this programming without imposing content, a choice that supported community rights.

18. Textiles Mónica Urquijo

- MIDAS

Atlántico Arts and crafts

This project was listed as a success story from MIDAS, but the evaluation found that its fortunes had changed since the end of the program. Afro-Colombian women were trained and supplied to create crafts for retail sale; the workers were women heads of households, some of them victims of forced displacement and violence. The company was designed to be profitable and to contribute materially to its participants' quality of life, but these have not been sustained for the long term. Cultural differences appeared to have played a part of the failure, as the majority Afro- Colombian producers left the project together.

PROJECT BACKGROUND

The U.S. Government (USG) made a substantial commitment through Plan Colombia (2000- 2005) to reduce illicit crop production and promote a culture of legality. Despite these interventions and GOC efforts, Colombia continued to struggle with the production of illicit crops. The GOC and USG together designed a three-pronged strategy to meet the challenge:

Interdiction, Eradication and Alternative Development (AD). The latter was intended to improve social and economic conditions in areas with illicit crop production, thereby empowering and enabling small producers to reject illicit crops. AD sought to stimulate licit and viable economies, expanding legitimate livelihoods and promoting sustainable economic development, particularly in rural areas vulnerable to illicit crops.

It was in this context that USAID introduced two large multifaceted AD programs to work in challenging environments where illicit activities, population displacement, violence and environmental challenges were potential, if not always active, threats. The projects tried to improve conditions for rural citizens through productive projects; community participation;

social infrastructure development; forestry projects; support to agribusinesses, micro-enterprises, small- and medium-sized enterprises (SMEs); strengthening municipal governments; improving access to credit; and public policy development. The programs – designed to respond to a key lesson learned in the field: people in isolated communities with little or no state presence and limited economic options are more likely to cultivate illicit crops1 -- were as follows:

• The Areas for Municipal-Level Alternative Development (ADAM) project sought to create a network of municipalities and communities committed to legality. Its goals were to strengthen the legitimacy of the state and to provide tangible incentives to local populations to participate in the development of their own communities. Anticipated results included a culture of zero illicit crops and improvements in the quality of life or marginalized populations in rural areas.

• The More Investment for Sustainable Alternative Development (MIDAS) project worked with the private sector, agribusiness, forestry and SME participants, and regional-level operators to reduce the production of illicit crops and create sustainable economic activities through agribusiness, commercial forestry, SMEs and Policy.

ADAM (2005-2011) used a territorial approach, in which productive communities were targeted for agricultural interventions, citizen participation, municipal governance, social infrastructure works, social organization strengthening, and access to state programs in some 75 municipalities.

ADAM targeted illicit activities through integrated economic, political and social approaches to agricultural production, social infrastructure, community participation and local governance.

The following figures – claimed by ADAM project records – provide a sense of scale: the agriculture component reports benefits to over 50,000 families, supporting 108,000 hectares, while generating 66,000 jobs and $51 million in sales. The social infrastructure component

1 TetraTech ARD, 2011. Areas de Desarrollo Alternativo Municipal (ADAM); Final Report (2005-2011). Preface. All ADAM results citations in this introduction are from this work.

Post-implementation evaluation of the ADAM and MIDAS programs – Draft report 6 reports 1822 projects, valued at $42 million, benefiting 38,000 families. The project cites 1,926 social organizations strengthened.

MIDAS (2006-2011) employed a demand-driven model to promote public and private alliances to provide income and employment in six geographical corridors. The goals were to eliminate illicit crop planting, promote competitiveness, improve social and economic policies and strengthen institutions. Many Colombian organizations worked in partnership with the programs to extend the influence of programming, ensure relevance, and work toward sustainability.

Project records claim leveraging of $571 million from public and private sectors benefiting 280,000 families; 164,000 new hectares planted and 260,000 new jobs.3 MIDAS also worked at the national level with significant contributions to policy and institutional reforms

2 This figure differs from that (220 projects) provided by ARD, but both figures come from the implementer.

3 TetraTech ARD, 2011. Mas Inversion para el Desarrollo Alternativo Sostenible (MIDAS); Final Report. All MIDAS results citations in this introduction are from this work.

Post-implementation evaluation of the ADAM and MIDAS programs – Draft report 7

EVALUATION DESIGN

Evaluation Methods The evaluation team used both quantitative and qualitative data as primary sources, as well as secondary documentary sources. Full evaluation design specifications, including detail on methods, sample, fieldwork and analyses are included in Annex II. Quantitative methods included a telephone survey of small- and medium-enterprises, forestry and agribusiness participants in MIDAS. The team applied a quantitative checklist in observing infrastructure projects in the field, along with a short questionnaire.

Qualitative methods began with desk and document review, and early interviews with USAID, implementer staff who were still available in Bogota, and related GOC partners. Further interviews were conducted in Bogota during December and January to cover additional topics, including MIDAS policy interventions and microenterprise programming. Interviews and focus groups were used extensively in field research. ADAM and MIDAS were large programs ($369,000,000 between the two) with a variety of field approaches across the country and within a given region. As a result, field teams were trained to pursue different actors within given alliances and projects at different levels. Researchers probed thematic areas beyond those found in the instruments, in accordance with the variety of roles and relationships represented by the different levels of participants, at vereda, municipality, project, SME, popular-level association, second-level association and operator levels. On rare occasions, when sufficient numbers of beneficiaries or other respondent types were available, the field teams conducted focus groups.

Where certain cases showed particular best practices or lessons learned, especially in cases of public-private partnerships, the field teams attempted multiple interviews with subjects from various perspectives on the implementation. These were included in a set of 18 case studies produced for the evaluation, three per geographic team. Please see Annex V for the full studies.

Sampling The team selected a purposive or convenience sample by geography, with the intention to cover Mission-prioritized activity sets. In Cauca, an advance team piloted the instruments. A summary of the sample is shown below in Figure 2.

Figure 2. Summary of the site visit evaluation sample Departments 13 Municipalities 57

Men Women Total Interview informants 111 70 181 Focus group participants 46 22 68

Totals 157 92 249

Through the pre-fieldwork interviews in Bogota, the team assembled lists of likely candidates for site visits from both accessing databases and other resources (SMEs, infrastructure project list, success stories, project one-pagers) and from respondent recollection and recommendations. The fieldwork required extensive use of the snowball sample technique once in the field.4 Most

4 Snowball sampling (or chain-referral sampling) is a non-probability sampling technique where existing study subjects support recruitment efforts for future subjects, from among their acquaintances, and in particular among known participants. (From Wikipedia’s definition, at http://en.wikipedia.org/wiki/Snowball_sampling) The limited timeframe for the evaluation and the fact http://en.wikipedia.org/wiki/Snowball_sampling

Post-implementation evaluation of the ADAM and MIDAS programs – Draft report 8 contact information in implementer databases was no longer valid; capture rates ranged from 10- 18 percent across the regions. As a result, the field teams pursued cases through project coordinators who were still in the field, through the few contacts that remained valid, or through going directly to communities that were recorded as having had a project, and asking for knowledgeable informants.

The summary of the quantitative survey samples is shown in Figure 3 below.

Figure 3. Summary of the quantitative survey sample Universe Sample Notes SME, agribusiness and forestry projects survey with MIDAS beneficiaries 473 69 Low capture due to outdated telephone contact information

Observations of ADAM infrastructure works 2205 42 Selected within geographic sites to be visited

Beneficiaries in communities with ADAM infrastructure projects N/A 44

Outdated contact information on project participants; convenience sample where contact information failed

Limitations and Gaps As the evaluation was carried out over three years after the end of the programs, with longer gaps since some activities had ended, the sample is limited to those respondents for whom contact data were available and up to date, or those who could be contacted through a snowball sample.

This affected access to all types of respondents: small and medium enterprises, agricultural and forestry producers and commercial operations, community associations, implementing staff members, and municipal and other government partners. The sample, then, is skewed toward those that have the same contact information. Such respondents are more likely to have remained in the same place (while missing displaced persons and those with other kinds of transience), to have maintained financial solvency (while missing those with financial challenges), to have continued in similar work (while missing those with labor condition transience), to have closer connections with other projects through whom the evaluation team could find them (while missing isolated cases), and so on. It is also worth noting that carrier competition and cell subscribers’ preferences have resulted in frequent changes to both personal and business lines, for reasons neutral to the evaluation. In the case of municipal officials involved with ADAM projects, few were found to still be in office or accessible through previous contact information.

The issue of expired contact information also affected the telephone survey completion rate, which was approximately 15 percent of MIDAS’ database of agribusinesses, small and medium enterprises, and forestry enterprises.

Because of these limitations on both the qualitative and quantitative samples, no results of the evaluation can be taken to be representative of the wider population involved.

that the programs ended years ago, limited the access to informants. By implementing the snowball method we were able to consult informants in all regions and we covered the sector.

5 This figure differs from that (182 projects) in ARD’s final report, but both figures come from the implementer.

Post-implementation evaluation of the ADAM and MIDAS programs – Draft report 9

FINDINGS, CONCLUSIONS & RECOMMENDATIONS

This section presents a synthesis of the field research and analyses of the evaluation data. It is organized in sections based on the key themes from among the ADAM and MIDAS activity sets under study. These grounded data, analyses and recommendations answer the guiding questions of the evaluation. The final section is a user’s guide for ‘connecting the dots’ between the evaluation results and upcoming programming, particularly in light of a possible peace process.

Because the fieldwork was extensive, covering several components of two substantial programs, the bulk of the findings – empirical data points – are shown in Annex I – Findings, Conclusions and Recommendations, in tabular form. This Annex is useful for the reader looking to plumb a given subject in more depth, as it contains the full range of findings on each key theme. This table also shows the relationship between conclusions and the findings on which they are based, as well as the recommendations that then emerge.

At the same time, the main body of the report provides a targeted review for readers who want to understand key themes across the areas of inquiry of the evaluation, in a concise and accessible way. The User’s Guide to the report, the final report chapter, orients the user in to how to apply the evaluation’s results, in the context of Colombia’s peace process and the Mission’s CDCS process.

The Conclusions section of each Key Theme emphasizes the how and why of the findings and these elements are underlined in the text of the Conclusions. Recommendations for key themes are also cross-referenced by the array of key themes to which they are related, and arrayed in a table by key theme at the end of this section. Finally, case studies are referenced in the text to indicate the detail that can be found in Annex V with eighteen program case studies.

A final annex that can help readers understand these key themes is Annex IV, Getting to Answers. As the evaluation questions for this study were unusual in their format – a matrix of guiding principles and sectoral questions – the evaluation team designed the evaluation around the key themes, below, and how they answered both the guiding principles and the sector-based questions. The key themes emerged from a facilitated process in the evaluation team’s extended team planning meetings.

1. Key Theme: Associativity

Findings

a. Associativity

In response to the evaluation fieldwork, a range of organizations and projects highlighted their experiences with associativity. In Bolivar, ADAM’s Aproaca project helped the organization to form, strengthen production, create citizen oversight, and remake the social fabric of their community (Please see Case Study (CS) 1, Annex V). Indigenous community dairy project respondents realized the value of associativity through implementing an ADAM project in their organization. The leader of that association and other association representatives emphasized the lack of associativity prior to these projects, with individuals working in isolation and mistrust

(CS 3).

Post-implementation evaluation of the ADAM and MIDAS programs – Draft report 10

A social worker in Cauca who worked with various ADAM projects called the associativity work “added-value,” saying that the feeling of belonging to the community was a very important result of building sewers, schools, or other infrastructure projects jointly.

The infrastructure observation and survey results include one infrastructure project that failed after ADAM staff left. A failure in an ADAM productive project in Cauca occurred when a new association dissolved for lack of capacity to resolve conflicts over resources and leadership. Most infrastructure projects (34 of 42 that were observed), however, continued to be cared for by a community organization.

b. Community contributions

Responses to the infrastructure survey and the observations themselves indicate a correlation between community participation in design, construction, management or maintenance, and the current physical state of the project. A respondent who led an ADAM school cafeteria project noted the importance of linking the associations to the projects through their contributions:

materials, labor and committee participation. “That’s the way they buy into the project,” he noted, “though the work was hard, excavating and leveling the ground for the cafeteria.” He said that associativity was particularly challenging – and important – for communities of displaced people like theirs.

c. Prior consultation and the consultative process

The ADAM design called for a consultative process with potential beneficiaries prior to all activities with communities and associations. For Afro-Colombian and indigenous communities, this process has an official name, “prior consultation,” reflecting its status in Colombian regulations. In a great many visits to communities and projects, this consultation process was reported to have been essential and very appreciated – including in communities and activities without Afro-Colombian or indigenous participants. A great number of positive experiences with this consultative process were encountered in the field, with a small but important number of negative ones that provide lessons learned.

The Aprocasur association debated with ADAM on which new crops and products to adopt.

Local producers with generations of experience said they knew better than ADAM staff what they wanted to plant and what would work well (CS 2). They chose cacao, and ADAM supported that decision, empowering them and building ownership.

Small producers in Cauca challenged the technical assistance (TA) they received on avocado planting, noting that the plant material provided was already past its prime, and that soil and rainfall conditions were not appropriate for the plants they were being offered. Nevertheless, since avocado was new to the area, the TA’s presumed expertise was followed. The planting failed and the producers were left indebted, having taken out loans to increase their planting and yield. TA and credit were provided by a second-level organization, described in the case study annex (CS 14).

Respondents appreciated a “bottom-up” approach to projects, in interview after interview. A participant in a coffee project in Huila appreciated that he and his family got to say what they needed, and see that reflected in the project that was ultimately agreed upon. He also noted: "The [GOC] program Rural Opportunities offered less money but there was more of a chance to work directly with the technical assistance [than there was with MIDAS].”

d. Participatory management and maintenance

Post-implementation evaluation of the ADAM and MIDAS programs – Draft report 11

Of 44 infrastructure survey respondents, 73 percent reported extensive community participation in managing project resources.6 Thirty respondents (of 40 valid responses, or 75%) said that the community provided maintenance after the project ended. Two noted, however, that these maintenance services were insufficient to maintain the technological aspects of their sewer and aqueduct systems. Nearly four-fifths (79.5%) of responses indicated that the facilities were in good shape at present; paint was the most common characteristic reported to be in a poor state of repair. Nearly all (95%) of respondents said that the facility was still in use for its original purpose. Most (79%) reported that a community organization existed to maintain the work, and only one respondent reported that there had once been such an organization but that it had disbanded after the facility was finished. More than three-quarters (77%) said that the community was still satisfied with the result. Five respondents (11%) reported satisfaction but also problems for continued functioning.

A community in Nariño identified the project they wanted to undertake – a comfortable and well-supplied school for their children’s welfare, development and learning – and then worked together to meet ADAM’s requirements for their contribution. They carried out raffles and a bingo night in order to provide the school with a small library with new books and an LCD television. They now carry out ongoing maintenance of the school and its grounds and report that the school is the pride of their community, one which neighboring communities want to emulate.

Conclusions

• Developing associativity in ADAM and MIDAS projects was a major achievement for the programs. Though it was not successful in every activity visited, it was quite frequent and respondents cited it as a major change. It is especially noteworthy in conditions of displacement, poverty and the presence of armed actors. Reconstructing social fabric required the individuals and communities establish – or re-establish – their life projects, work collectively, and involve all generations. This contributed to a shared identity, shared enterprise, and to rooting the citizens again in their communities.

• When ADAM and MIDAS worked with established organizations, the learning period was shorter, producing stronger community appropriation and sustainability. Greater associativity in MIDAS projects was correlated with greater community agreement on processes of agricultural production and collective strategies to face market requirements.

• ADAM projects strengthened associations through the activities of the citizens’ oversight and management committees, and through capacity building in accounting and organizational skillsets. Community contribution in these projects was key to feelings of ownership. Where there was greater participation from communities in terms of funds, land, labor and risk, there was also greater local commitment to sustainability.

• Deciding which project to undertake empowered participants and increased their commitment to the project and to the contributions they would have to make to it. Early consultation at decision-making levels allowed local knowledge of agricultural, economic, infrastructural or conflict-related contexts to be recognized and incorporated into projects.

6 These data are not representative of all ADAM social infrastructure projects, as the sample was purposive rather than random.

These findings cannot therefore be generalized, nor assumed to apply to all such projects. However, the data provide strong indications of strengths, lessons learned, and the implementation model utilized, including its associativity.

Post-implementation evaluation of the ADAM and MIDAS programs – Draft report 12

• Some implementing staff members had better results than others in prior consultation. This may have reflected the varying situations they were in as much as their individual capabilities

– but the communities needed to perceive that implementation was sensitive to their differences and needs, in order to garner their trust.

• Involving communities in administering resources generated credibility and participants assumed responsibility. Where community members and their organizations were empowered to criticize plans, decision-making and participation were considered genuine.

Where participation was weakest in terms of community planning and decision-making, communities experienced conflicts in their expectations and project processes.

Recommendations

1. Continue to build associativity into productive and infrastructure projects with bottom-up programming with genuine community participation. Build time for this challenging undertaking into timelines, particularly with returnee communities, with new or newer associations, and when working across multiple communities. Diagnose and support locally necessary TA to reach associativity goals and build relevant capacity. Include detailed TA and training content on the benefits of associativity tailored to communities, using real-world examples.