Mid-Term Evaluation Report for the USAID Reforestation Project-USAID 2021.pdf

i | USAID/ HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV

USAID/HAITI MID-TERM EVALUATION

OF THE REFORESTATION PROJECT

FINAL REPORT

Haiti Evaluation and Survey Services Project

May 2021

PHOTO CREDIT: USAID / HAITI

USAID.GOV USAID/HAITI REFORESTATION PROJECT FINAL REPORT | ii

USAID/HAITI MID-TERM EVALUATION

OF THE REFORESTATION PROJECT

FINAL REPORT

Date: May 21, 2021

Submitted to USAID/EGAD

Contract Number: CA AID-521-A-17-00002

This publication was prepared independently by Social Impact, Inc. at the request of the United States

Agency for International Development

Authors: Cynthia Berning, Team Leader

Philippe Phanol, Deputy Team Leader

Contact: Salima Mutima, Chief of Party

2300 Clarendon Blvd, Suite 1000

Arlington, VA 22201, USA

DISCLAIMER

The author’s views expressed in this publication do not necessarily reflect the views of the United States

Agency for International Development or the United States Government.

iii | USAID/ HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV

ABSTRACT

The United States Agency for International Development (USAID) Haiti requested Social Impact’s (SI)

Evaluation and Survey Services (ESS) project to design and implement an independent mid-term evaluation of the USAID Reforestation Project (URP) in Haiti, a Project implemented by Chemonics

International in partnership with The National Cooperative Business Association Cooperative League of

USA International (NCBA CLUSA) and the International Center for Tropical Agriculture (CIAT). This evaluation aims to inform USAID/Haiti about possible mid-course corrections and future program orientation of the URP, whose objective is to reduce the threat to targeted forests and increase tree cover.

This midterm evaluation answers evaluation questions (EQs) related to (1) the effectiveness of two implementation mechanisms (direct investment vs. grants), (2) if the Project is improving beneficiaries’ resilience1 in the face of natural and economic shocks, (3) if the Project is integrating youth and gender, and (4) whether the Project has laid the foundation for sustainability.

The Project's two implementation mechanisms each have their advantages and disadvantages that should be strategically weighed in future project development, the resilience activities have increased income but likely only in the short term, that gender and youth integration has not been a strong focus of the

Project, and that few if any of the activities will lead to a sustainable reduction in deforestation

1 USAID defines resilience as the ability of people, households, communities, countries, and systems (social, economic, ecological) to mitigate, adapt to, and recover from shocks (including co-variates such as drought and floods…) and stresses

(climate changes, population pressure…) in a manner that reduces chronic vulnerability and facilitates inclusive growth.

USAID.GOV USAID/HAITI REFORESTATION PROJECT FINAL REPORT | iv

TABLE OF CONTENTS

ABSTRACT III

TABLE OF CONTENTS IV

TABLES AND FIGURES V

ABBREVIATIONS AND ACRONYMS VI

EXECUTIVE SUMMARY 1

I. INTRODUCTION 7

I1. EVALUATION DESIGN 12

III. FINDINGS AND CONCLUSIONS 23

IV. RECOMMENDATIONS 44

V. EVALUATION MATRIX 47

VI. EVALUATION MANAGEMENT 50

VII. EVALUATION DELIVERABLES AND TIMELINE 53

ANNEX A. SCOPE OF WORK 55

ANNEX B. DATA COLLECTION PROTOCOLS 61

ANNEX C. DATA COLLECTION TOOLS 62

ANNEX D. DOCUMENTS REVIEWED 81

ANNEX E. LIST OF DATA COLLECTION EVENTS 83

ANNEX F. URP SUB-AWARDEES 87

ANNEX G. TREE SEEDLINGS PRODUCED AND TRANSPLANTED 91

ANNEX H. TRAININGS CONDUCTED IN 2018-2020 92

ANNEX I. WOMEN’S ORGANIZATIONS ASSESSED IN FY2018 96

ANNEX J. EXAMPLE OF INFORMATION NEEDED REGARDING SEX-

DISAGGREGATED NUMBER OF PARTICIPANTS IN TRAININGS 97

ANNEX K. SIGNED DISCLOSURE OF CONFLICT OF INTEREST FORMS FOR ALL

EVALUATION TEAM MEMBERS 98

v | USAID/ HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV

TABLES AND FIGURES

Figure 1. URP RESULTS FRAMWORK (2018)

Figure 2. USAID REFORESTATION PROJECT THEORY OF CHANGE

TABLE 1. LIST OF FGDS CONDUCTED

TABLE 2. STAKEHOLDERS INTERVIEWED DURING FIELDWORK

TABLE 3. NUMBER AND DISTRIBUTION OF KIIs/GIs BY STAKEHOLDER GROUP AND EQS

ADDRESSED

TABLE 4. URP'S OUTPUTS OF ENVIRONMENTAL SERVICES DELIVERED PER TYPE OF

IMPLEMENTATION MECHANISM

TABLE 5. URP'S OUTPUTS OF SEEDLINGS PRODUCED AND TRANSPLANTED PER TYPE OF

IMPLEMENTED MECHANISM

TABLE 6. PROS AND CONS OF DIFFERENT IMPLEMENTATION MECHANISMS

TABLE 7. DECISION MATRIX FOR GRANTS VS. DIRECT INVESTMENT

TABLE 8. EVALUATION MATRIX

TABLE 9. EVALUATION TEAM MEMBER ROLES AND RESPONSIBILITIES

TABLE 10. DELIVERY SCHEDULE

TABLE 11. EVALUATION SCHEDULE

TABLE 12. EVALUATION DESIGN MATRIX

TABLE 13. KIIs

Table 14. GIs

Table 15. FGDs

TABLE 16. URP SUB-AWARDEES

Table 17. SEEDLINGS PRODUCED AND TRANSPLANTED AS OF THE END OF FY20

TABLE 18. TRAININGS CONDUCTED IN 2018-2020

TABLE 19. WOMEN'S ORGANIZATION ASSESSED IN FY2018

TABLE 20. TRAININGS CONDUCTED DURING FY2020 Q3

USAID.GOV USAID/HAITI REFORESTATION PROJECT FINAL REPORT | vi

ABBREVIATIONS AND ACRONYMS

3BNMP 3-Bays National Marine Park

AAF Association des Apiculteurs de Ferrier

ADS Automated Directive System

ANCASA Antillean Canning Societé Anonyme

AOR Agreement Officer’s Representative

APDK Asosiyasyon Pechè pou Devlòpman Karakòl

APM Asosyasyon Pechè Madras

ASEC Assemblée des Sections Communales

ATL Assistant Team Leader

CACOPA Cadre de Concertation pour la Promotion de Apiculture

CART Coopérative Agricole Rabouré de Terrier Rouge

CASB Centre pour l’agriculture Saint Barnabas

CASEC Conseil d’Administration de la Section Communale

CBO Community-Based Organization

CFAIM Centre de Formation en Aménagement Intégré des Mornes

CHASE Collectif Haitien pour l’Avancement du Suivi et de l’Evaluation

CIAT International Center for Tropical Agriculture

CMBP Caribbean Marine Biodiversity Program

COAIT Community Options Analysis and Investment Toolkit

COP Chief of Party

COR Contracting Officer’s Representative

CRDD Centre Rural de Développement Durable

DAI Development Alternatives Inc.

DCOP Deputy Chief of Party

DDA Direction Départementale Agricole

DDE Direction Départementale de l’Environnement

DEC Development Experience Clearinghouse

DEED Développement Economique pour un Environnement durable

DI Direct Investment

DO Development Objective

EGAD Economic Growth and Agricultural Development

EQ Evaluation Question

EQUI Evaluation Quality, Use, and Impact

ESS Evaluation and Survey Services

ET Evaluation Team

FBO Faith-Based Organization

FECCANO Fédération des Coopératives Cacaoyères du Nord

FCR Findings, Conclusions, and Recommendations

FGD Focus Group Discussion

FoPRoBim Fondation Pour la Protection de la Biodiversité Marine

FY Fiscal Year

GAO Groupement Agricole Ouvray

GESI Gender and Social Inclusion

GI Group Interview

GOH Government of Haiti

HQ Headquarters

IDDH Initiative pour le Développement Durable d’Haiti

IPTT Indicator Performance Tracking Table kha Kilohectare vii | USAID/ HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV

KII Key Informant Interview

LOE Level of Effort

LPG Liquefied Propane Gas

M&E Monitoring and Evaluation

MARNDR Ministry of Agriculture and Natural Resources

MCAD Mouvement Coopératif Agricole de Dubourg

MEL Monitoring, Evaluation, and Learning

MMK Mobile Restaurant

MOU Memorandum of Understanding

MSME Micro, Small, and Medium Enterprise

MSPJ Marais Salant Partout de Jacquesil

MT Metric Ton

NCBA/CLUSA National Cooperative Business Association/Cooperative League of USA

NGO Non-Governmental Organization

NRM Natural Resources Management

NUPAS Non-U.S. Organization Pre-Award Survey

ODL Organisation pour le Développement de Labadie

OPPBL Oganizasyon Pechè pou Pwoteksyon Bodmè Limonade

PA Project Assistant

PAC/VIAHSA Paul Agribusiness and Culinatity/Village Apicole Haitienne S.A

PAP Port-au-Prince

PD Project Director

PEA Political Economy Analysis

PM Project Manager

QA Quality Assurance

RCCL Royal Caribbean Cruise Line

RECOCARNO Réseau des Coopératives Caféières de la Région Nord

RFEO Rassemblement des Femmes Engagées de Ouanaminthe

SI Social Impact

SOW Scope of Work

SWMC Sub-Watershed Management Committee

TL Team Leader

TOC Theory of Change

UNEPH Université Episcopale d’Haiti

URP USAID Reforestation Project

USAID United States Agency for International Development

1 | USAID/ HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV

EXECUTIVE SUMMARY

The United States Agency for International Development (USAID) Mission in Haiti requested Social

Impact’s (SI) Evaluation and Survey Services (ESS) project to design and implement an independent mid-term evaluation of the USAID Reforestation Project (URP) in Haiti. The Project is implemented by

Chemonics International in partnership with the National Cooperative Business Association

Cooperative League of USA International (NCBA CLUSA) and the International Center for Tropical

Agriculture (CIAT).

The URP’s goal is to increase forest and tree cover in the targeted sub-watersheds and other strategic areas, while contributing to the resilience of targeted populations to economic and natural shocks and long-term stresses for their improved well-being in Haiti’s North and North-East regions. In order to fulfill this vision, a five-year, US$39,305,099 project was designed and approved with a start date of

September 1, 2017. The URP is scheduled to close in August 2022.

The URP aims to achieve four results, each with four intermediate results: 1) threat of deforestation reduced; 2) resilience to economic and natural shocks improved; 3) tree cover in targeted areas increased; and 4) environmental governance and coordination improved.

This Final Report outlines the evaluation team’s (ET) understanding of the evaluation scope of work

(SOW), provides background on the Project, explains the evaluation’s methodology (adapted to current

COVID-19-related emergency measures to protect public health), presents findings based on the ET’s review of Project documentation and fieldwork efforts, and highlights recommendations for the final two years of the project and future USAID programming.

EVALUATION PURPOSE AND QUESTIONS

The purpose of this mid-term performance evaluation is to inform USAID/Haiti about possible mid-course corrections and future program orientation of the URP. The primary audience for this final evaluation includes USAID/Washington, USAID/Haiti, Chemonics (the implementing partner), the grantees and sub-grantees, and the Government of Haiti (GOH).

This midterm evaluation answers the following evaluation questions (EQs):

1. Effectiveness: What are the strengths and weaknesses of the implementation mechanisms (direct investment vs. grants) as utilized separately and together to promote improved natural resource management?2

2. Resilience: To what extent and in what ways is the Project improving beneficiaries’ resilience3 in the face of natural and economic shocks?

3. Youth and Gender: To what extent and in what ways is the Project integrating youth and gender?

4. Sustainability: After two and half years of implementation, to what extent and in what ways has the Project laid the foundation for sustainability in terms of beneficiaries’ improved approaches toward natural resources management through the adoption of principles and practices promoted by the Project?

2 Natural resources management (NRM) is the management of resources such as land, water, soil, plants, and animals, with a particular focus on how management affects the quality of life for both present and future generations (stewardship).

3 USAID defines resilience as the ability of people, households, communities, countries, and systems (social, economic, ecological) to mitigate, adapt to, and recover from shocks (including co-variates such as drought and floods) and stresses

(climate changes, population pressure) in a manner that reduces chronic vulnerability and facilitates inclusive growth.

USAID.GOV USAID/HAITI REFORESTATION PROJECT FINAL REPORT | 2

DETAILED METHODOLOGY

The ET used a primarily qualitative evaluation design, consisting of document review, 27 key informant interviews (KIIs), 5 Group Interviews (GIs) and 20 focus group discussions (FGDs).

The ET applied several analysis methods (content, trend, gap, and gender) to provide evidence for the evaluation’s findings and conclusions. The team disaggregated data collected through KIIs, GIs, and FGDs by sex and analyzed for effects on different participant segments. Evaluation challenges included Internet connectivity, difficulty in reaching some key contacts, and low levels of knowledge of the project and its achievements among many respondents due to delays in project implementation that resulted in many activities that were only recently begun. The ET conducted fieldwork from April 12-30, 2021. Upon final approval of the report, ESS will submit the report to the Development Experience Clearinghouse (DEC) for dissemination.

FINDINGS RELATED TO THE EVALUATION QUESTIONS

EQ-1: WHAT ARE THE STRENGTHS AND WEAKNESSES OF THE IMPLEMENTATION

MECHANISMS (DIRECT INVESTMENT VS. GRANTS) AS UTILIZED SEPARATELY AND

TOGETHER TO PROMOTE IMPROVED NATURAL RESOURCE MANAGEMENT

(EFFECTIVENESS)?

FINDINGS

The URP’s focus in the beginning on implementation through grants elicited a limited response from eligible local organizations. Concerned that this approach would not achieve the Project’s targets, the focus shifted to more direct investment starting in FY19.

In FY2018, according to the FY2018 annual report, to implement the Project’s resilience-focused and reforestation activities the URP only used a grant mechanism. Despite the project officially launching in

September 2017, the Project signed only one grant agreement in 2018 and was seriously behind in reaching its targets for Years 1 and 2. With the shift to direct investment, the establishment and/or rehabilitation of forests and woodlots, agroforestry production, the number of people receiving the

Project’s livelihood-related co-benefits and seedling production was able to ramp up substantially even though by the end of FY20, the actual number of seedlings planted still did not meet the Project’s targets.

Project and USAID staff stated that there were pros and cons to both mechanisms, with no method clearly preferred across the board. There were differing opinions about whether one or the other mechanism was better for the Project, with many respondents saying that both were important and appropriate within the Project because they accomplished slightly different goals – Direct

Investment has allowed for a more systematic and large-scale roll-out of activities to reach targets and has resulted on more trees in the ground in a shorter amount of time, while grants facilitated capacity building among community organizations and thus could contribute more to sustainability.

Most sub-awardee respondents stated that the grant process was too long and difficult, with organizations selected through a call for proposals and with reimbursements taking too long, which made it difficult to reach targets.

A common related experience of sub-awardees has been that the Project promised equipment, materials, or other assistance that came very late or still has not come.

3 | USAID/ HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV

CONCLUSIONS

Given the short timeline, substantial delays, and the aggressive targets of the project, the move to direct investment was logical. Grant management, including capacity building, will take too long and likely face too many delays to be able to successfully complete sub-grant activities before the end of the Project.

Given the time required for processing grant agreements and working with organizations, the project does not have time to pursue further grants.

Even with fewer delays at the beginning of the project, the grants mechanism as it was implemented was unlikely to help the Project reach its targets. The arrangement of reimbursing organizations after activities are completed may not be a feasible approach in a resource-poor place like these targeted regions of Haiti.

A focus on grants, with organizations selected through a call for proposals, led to the scattered and sometimes not-logically connected assortment of activities. This strategy ensures that the Project is not attempting to implement an activity that is not of interest in a particular community. On the other hand, it leads to a scattered approach of disconnected individual activities that do not always have a logical connection to the project’s objectives. With the move to direct investment, the project became more able to focus on the specific activities that would lead to the achievement of the main objectives of the project – namely the planting of 4 million trees.

Delays on the part of the Project in providing promised materials have had negative impacts on activities. The ET did not learn why these delays occurred, but they were a frequent enough experience among grantees to suggest a systemic issue with Project procurement procedures rather than a one-time or occasional problem due to specific unforeseen circumstances.

EQ-2: TO WHAT EXTENT AND IN WHAT WAYS IS THE PROJECT IMPROVING

BENEFICIARIES’ RESILIENCE IN THE FACE OF NATURAL AND ECONOMIC SHOCKS

(RESILIENCE)?

FINDINGS

The focus of the Project shifted from primarily reforestation activities to a broader collection of activities intended to improve the resilience of beneficiaries by increasing their incomes and improving their management of livestock, but the Project’s indicators do not reflect that focus. Nearly every stakeholder who was interviewed agreed that resilience activities were a necessary part of a reforestation project, but many people stated that the resilience activities that were part of the project may have benefited people, but that the activities were too few and spread too thinly among communities to make a big difference.

The resilience activities that were most frequently mentioned as effective were the livestock-focused activities, followed by the bee-keeping activities.

The Project implemented instant infusion of cash, free tools, plants, and seeds. The most-cited benefit of the Project was the cash for work strategy it employed through both the grants and direct investment mechanisms.

CONCLUSIONS

While the various resilience activities individually could be beneficial, the project lacked a cohesive strategy (or struggled to effectively implement a strategy) that would ensure enough of an income increase to prevent further deforestation. Deforestation is a very

USAID.GOV USAID/HAITI REFORESTATION PROJECT FINAL REPORT | 4

complex issue with a lot of interrelated causes, and people will continue to cut down trees as long as they are poor and have no other alternatives to earning enough income. The activities most likely to lead to improved resilience are the livestock-focused ones.

Cash for work and provision of free tools, plants, and seeds affected beneficiaries’ livelihoods immediately (though temporarily). While it is unknown whether the longer-term income generating activities will permanently raise incomes for beneficiaries, it is clear that this instant infusion of cash, free tools, plants, and seeds did have an immediate impact.

EQ-3: TO WHAT EXTENT AND IN WHAT WAYS IS THE PROJECT INTEGRATING

YOUTH AND GENDER?

FINDINGS

It is unclear whether and to what extent gender/youth were considered during URP’s initial/design phase. URP has sex- and age-disaggregated indicators, but its gender/youth reporting is inconsistent. Beyond participation numbers for women, no partners are tracking any other indicators of inclusion.

CONCLUSIONS

Gender targets and quotas have been easily met by partners, and women appear to be active in decision-making, not just token members. To minimize potential self-censorship in mixed-gender FGDs, the ET conducted three all-female FGDs, and the same message was relayed in these: that women are active participants and involved in decision-making and were active before the

Project as well. It is difficult to assess whether the Project’s approach to reaching women within the context of traditional gender roles in the sector was a positive way to ensure the equal participation of women or in fact reinforced potentially harmful, traditional social norms or inequalities.

EQ-4: TO WHAT EXTENT AND IN WHAT WAYS HAS THE PROJECT LAID THE

FOUNDATION FOR SUSTAINABILITY IN TERMS OF IMPROVED APPROACHES TO

NATURAL RESOURCES MANAGEMENT THROUGH THE ADOPTION OF PRINCIPLES

AND PRACTICES PROMOTED BY THE PROJECT (SUSTAINABILITY)?

FINDINGS

The Resilience Activity benefits that people most frequently cited were short-term. FGD participants spoke more about the benefits of paid labor through the Project, which will end when the

Project ends, than about benefits that will continue post-Project. While Project staff spoke positively about the sustainability of resilience activities such as beekeeping and livestock forage growing, USAID and GOH staff were less optimistic.

URP explored partnerships with the private sector to increase income-generating activities for its stakeholders but most of these conversations have stalled. This was the case for both

Royal Caribbean Cruise Lines (RCCL), which was unable to be reached, and Antillean Canning S.A.

(ANCASA), which was reached through a KII.

Sub-Watershed Management Committees are currently functioning but only with Project support.

5 | USAID/ HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV

The trees that have been planted on people’s private property that will contribute to their income are likely to be protected, but trees planted on public land where animals roam freely are not likely to survive or be cared for. The URP supported the finalization of municipal decrees designating reforestation areas, but these decrees are unlikely to have much effect on tree cutting.

CONCLUSIONS

Most resilience activities are not likely to be sustainable, given the short project timeline, lack of resources among farmers, and many project delays. Beneficiaries learned new practices and incorporated those practices into their agricultural activities but did so primarily because seeds, plants, and training were given to them for free and they were paid a daily wage to implement these practices on their own parcels. Sustainable behavior change takes time, with subsistence farmers especially tending to be risk-averse and often requiring several seasons of tangible evidence of benefits in order to enact new techniques. At most, beneficiaries will have experienced one or two harvests of new crops, or in the case of slower-growing trees, no visible benefits of the Project’s activities for several years. Pursuing new and unproven techniques is a lot to expect from people who are already food insecure and cannot take the risk of a potential lost harvest, even if there is the possibility of increased income using new techniques.

While the exercise of creating a watershed management plan may have some small benefits to the communities, the committees are unlikely to continue to function without support. Without an entity financing activity of the SWMCs, organizing meetings, and paying travel/per diem costs, the committee members themselves stated that they would not be able to continue to do anything after the project ends.

The Project’s limited work with clean cookstove promotion is limited and is unlikely to significantly lessen the demand for charcoal. The Project’s efforts in this domain are not going to significantly reduce the demand for charcoal in these two regions to spark the kind of landscape-level change that the Project aimed to achieve.

RECOMMENDATIONS

In the final two years of the Reforestation Project, Chemonics should as a first priority:

● Not sign any additional grant agreements at this point, focusing on completing what is underway.

● Understand why delivery of materials has been so slow and prioritize the fulfillment of outstanding promises.

● Prioritize finding a way to fence off areas from animals where trees are to be planted in order to protect the trees.

● Continue to employ as many women as possible in the direct investment activities and pay them at least a standard daily wage for their time.

Once these top priorities are met, Chemonics should aim to:

● Focus remaining efforts on the activities that have shown fast uptake, particularly the livestock-related ones.

● Focus on developing the private sector cooperatives and trade associations like FECCANO and

RECOCARNO because they have the best chance at providing market opportunities for agroforestry crops.

In future reforestation-focused projects, USAID/Haiti should:

USAID.GOV USAID/HAITI REFORESTATION PROJECT FINAL REPORT | 6

● Strategically evaluate how to effectively include both direct investment and sub-grants mechanisms in implementation, likely with direct investment for the majority of the main project, but using grants where local organization exist to support promising local projects.

● Require a more logical and cohesive strategy for increasing income to a level high enough to decrease deforestation in these poor areas.

● More specifically define the project’s objectives and intended outcomes around youth and gender beyond targeting female participants.

● Understand the different motivations for deforestation-causing behaviors among various sub-groups of the population and attempt to focus messaging and activities more specifically to these different demographics.

● Balance a results-based payments approach with realities on the ground.

● Focus tree planting efforts on places and species that people have strong incentives to protect.

7 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV

I. INTRODUCTION

The United States Agency for International Development (USAID) Mission in Haiti requested Social

Impact’s (SI) Evaluation and Survey Services (ESS) project design and implement an independent mid-term evaluation of the USAID Reforestation Project (URP) in Haiti. The Project is implemented by

Chemonics International in partnership with the National Cooperative Business Association

Cooperative League of USA International (NCBA CLUSA) and the International Center for Tropical

Agriculture (CIAT).

The URP’s goal is to increase forest and tree cover in the targeted sub-watersheds and other strategic areas, while contributing to the resilience of targeted population to economic and natural shocks and long-term stresses for their improved well-being in Haiti’s North and North-East regions. In order to fulfill this vision, a five-year, US$39,305,099 project was designed and approved with a start date of

September 1, 2017. The URP is scheduled to close in August 2022.

The URP aims to achieve four results, each with four intermediate results: 1) threat of deforestation reduced; 2) resilience to economic and natural shocks improved; 3) tree cover in targeted areas increased; and 4) environmental governance and coordination improved.

This Final Report outlines the evaluation team’s (ET) understanding of the evaluation scope of work

(SOW), provides background on the Project, explains the evaluation’s methodology (adapted to current

COVID-19-related emergency measures to protect public health), presents findings based on the ET’s review of Project documentation and fieldwork efforts, and highlights recommendations for the final two years of the project and future USAID programming.

This mid-term evaluation is taking place during the COVID-19 pandemic. In addition to its devastating public health consequences, the pandemic is causing major disruptions in daily life in nearly all countries across the globe, including Haiti. This final report describes how the ET took into consideration the pandemic when developing the evaluation planning and methodology and carrying out data collection, by conducting all KIIs, GIs, and FGDs remotely with minimal travel and face-to-face contact.

The scope of this evaluation covers the period from the start of the URP until end of FY 2020

(September 30, 2020). The URP is being implemented in the north/north-east of Haiti, and the evaluation will look at the different activities implemented in both Region 1 and Region 2.4

Region 1

● The sub-watersheds of Joli-Trou/Cormier and Bahon (Grande Riviere du Nord watershed) for which management plans were completed, and in the sub-watershed of Milot (Haut du Cap watershed) for which a management plan will be developed this year. The latter will integrate the direct investment activities ongoing in the communal sections of Perche-de-Bonnet and

Bonnet à l’Evêque;

● The mangroves area of the coastal zones of the Bay of Acul du Nord, the 3-Bays National

Marine Park (3BNMP) and their surrounding communities;

● The communal section of Bande du Nord (home to village Labadie and RCCL Private

Destination – Tourist Resort Labadee), as part of the Project’ strategic partnership with Royal

Caribbean Cruise Line; and

4 Workplan 2020 (priority zones, p. 18)

USAID.GOV USAID/HAITI REFORESTATION PROJECT FINAL REPORT | 8

● The geographic areas covered by the cooperative members of Fédération des Coopératives

Cacaoyères du Nord (FECCANO) and Réseau des Coopératives Caféières de la Région Nord

(RECOCARNO) for the cocoa- and coffee-based agroforestry production systems.

Region 2

● The sub-watersheds of Perches (Marion watershed), Gens de Nantes (Jassa watershed), and

Capotille (border area – Massacre watershed) for which management plans were completed, and in the sub-watershed of Haut Ouanaminthe – communal sections of Savane Longue and

Savane au Lait (Jassa watershed) for which a management plan will be developed this year; and

● The mangroves area of the Bay of Fort-Liberté and their surrounding communities.

EVALUATION PURPOSE AND AUDIENCE

The evaluation examines how new processes, mechanisms, and systems established by the URP have contributed to restoring environmental services, improving livelihoods, and building the resilience capacity of beneficiary communities through integrated reforestation interventions. Lessons learned and recommendations provided through the evaluation report will also guide how the Project is strategically, operationally, and sustainably progressing toward its objectives, including youth and gender inclusion in reforestation programming and governance.

The primary audience for this final evaluation includes USAID/Washington, USAID/Haiti, Chemonics

(the implementing partner), the grantees and sub-grantees, and the Government of Haiti (GOH).

This evaluation focused on the Evaluation Questions (EQs) below:

1. Effectiveness: What are the strengths and weaknesses of the implementation mechanisms

(direct investment vs. grants) as utilized separately and together to promote improved natural resource management?

2. Resilience: To what extent and in what ways is the Project improving beneficiaries’ resilience in the face of natural and economic shocks?

3. Youth and Gender: To what extent and in what ways is the Project integrating youth and gender?

4. Sustainability: After two and half years of implementation, to what extent and in what ways has the Project laid the foundation for sustainability in terms of beneficiaries improving approaches to natural resources management through the adoption of principles and practices promoted by the Project?

The full evaluation SOW is included in Annex A and detailed evaluation protocols in Annex B.

9 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV

BACKGROUND

DEFORESTATION IN HAITI

Deforestation has been a major concern in Haiti, adversely affecting land use practices, destroying biodiversity and ecological habitats, and making long-term investments in the lowlands vulnerable to natural threats (hurricanes, soil erosion, floods, and drought). In the North and North-East regions of

Haiti, trees are removed for agriculture, construction, and charcoal production, primarily to ensure day-to-day economic livelihoods and food subsistence in the face of major stressors and shocks, such as drought and lack of alternative income sources. According to Global Forest Watch, from 2002 to 2020, Haiti lost 72.1kha of tree cover, including 2.87 kha of humid primary forest, or 8.4 percent of the 2000 total tree cover. The total area of humid primary forest in Haiti decreased by 33 percent in this time period.5

URP PROJECT GOAL AND OBJECTIVES

The URP’s goal is to increase forest and tree cover in the targeted sub-watersheds and other strategic areas, while contributing to the resilience of targeted populations to economic and natural shocks and long-term stresses. The five-year, US$39,305,099 project was designed and approved with a start date of September 1, 2017 and is scheduled to close in August 2022.

The URP’s Results Framework includes one overarching goal to increase the foundations for resilience, stability, and inclusive growth. To contribute to this goal, URP aims to achieve four results—1) threat of deforestation reduced; 2) resilience to economic and natural shocks improved; 3) tree cover in targeted areas increased; and 4) environmental governance and coordination improved—each with four intermediate results.

Figure 1 presents the objective of the USAID Reforestation Project “restore the provision of environmental services in targeted forest and agroforestry areas in support of food security and/or resilience to economic and natural resources shocks” with a direct link to the USAID/Haiti Strategic

Framework for 2018 – 2020 and Results Framework goal “foundation for resilience, stability, and inclusive growth reinforced.”

5 https://www.globalforestwatch.org/dashboards/country/HTI https://www.globalforestwatch.org/dashboards/country/HTI

USAID.GOV USAID/HAITI REFORESTATION PROJECT FINAL REPORT | 10

Figure 1. URP RESULTS FRAMWORK (2018)

Source: Annual Report FY 2018, p.47

The Project’s direct beneficiaries include charcoal value chain actors, marketers, consumers of agroforestry products, smallholder farmers, and fisherfolk, as well as watershed residents and fuel-wood users. Their participation and behavior change are necessary to achieve results. In addition, the Project plans to build the capacities of sub-watershed management committees as well as provide training to municipalities.

The Project’s indirect beneficiaries include urban, coastal, and downstream residents; plains farmers;

hotel and tourism operators; and enterprises whose daily needs are met by forest and watershed environmental services. The Project aims to especially target women and youth as participants and beneficiaries.

URP’S THEORY OF CHANGE

URP’s theory of change (TOC) can be summarized as “tree cover will increase if the threat of deforestation is reduced, the incentive to plant and maintain new trees is increased, community resilience to economic and natural shocks is improved, and environmental governance and coordination is strengthened.” Figure 2 presents the Project’s TOC.

11 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV

Figure 2. USAID REFORESTATION PROJECT THEORY OF CHANGE

Source: Annual Report FY 2018, Project Document, page 6

The assumptions underlying this TOC are:

● Political stability and effectiveness of Project’s partner ministries: The partnership with and support of

Haitian ministries and government is critical to the Project’s success and sustainability. The

Project is aware of potential political instability and the possible effects not only on its institutional champions, partnerships, and buy-in, but also on Project operations and safety.

● No natural or ecological disasters: The Project’s sub-watershed management and agroforestry efforts have the potential to be greatly affected by natural and ecological events. The plan for sustainable impact is closely tied to the physical state of the target area.

● Security: The Project’s function and activities are dependent on security in the region. Haiti, including the northern region, is regularly affected by violent protests and other insecurity, so it is critical to consider security in developing and monitoring the exit strategy.

● Continuous leadership of grantees and local partners: Grantees and local partners play a critical role in the Project’s sustainability and impact. As such, their continuous leadership, buy-in, and increased capacity are important factors in the Project’s exit strategy and long-term impact.

URP SUMMARY PROJECT INTERVENTIONS

To achieve enduring results and outcomes, the Project tries to understand and address the primary drivers of deforestation in the region and builds on the findings from assessing the region’s economic, institutional, and cultural characteristics. This approach helps to identify the opportunities and challenges inherent in the Project’s implementation process in the area. The Project involves and works with government entities at the central and regional level to ensure institutional and policy ownership of its activities. It partners with local organizations (community-based, faith-based, non-governmental) and the private sector to implement initiatives that sustain the local economy while improving the livelihoods of targeted groups.

USAID.GOV USAID/HAITI REFORESTATION PROJECT FINAL REPORT | 12

To implement its activities, the Project uses two mechanisms: partnerships with the above-mentioned organizations through sub-awards and in-kind grants; and direct investments in collaboration with municipalities and other actors involved in community development. The main activities of the Project are:

● Promoting use of efficient cookstoves and liquefied petroleum gas as a strategy to reduce pressure on tree stands and increase tree cover in the targeted sites.

● Establishing community nurseries to produce forest, fruit, and mangrove seedlings for reforestation and rehabilitation of agroforestry systems.

● Establishing forest woodlots, fruit orchards, and agroforestry systems (mixed of fruit and forest trees and cash crops).

● Rehabilitation of degraded mangrove forests to protect the coastal communities and ensure conservation of mangrove ecosystems.

● Strengthening the productivity and resilience of livestock systems, through drought adaptation measures, deworming, forage production, and storage.

● Developing livelihoods and income diversification sources (beekeeping, short-term employment, agroforestry production, sustainable fishing) to help households build economic assets and cope with major stresses and shocks affecting food security in the region.

● Building the capacity of targeted groups and local governments to develop and implement sub-watershed management plans that promote productive land use, reduce disaster risks, and improve natural resource management governance.

I1. EVALUATION DESIGN

PHASE ONE PLANNING AND PREPARATION

This mid-term evaluation formally began with a client kick-off meeting on December 2, 2020. During the meeting, the TL, ATL, ESS Field Office staff, ESS Contracting Officer’s Representative (COR), USAID/Haiti technical team members who manage URP, and Chemonics team members, including the

Chief of Party (COP) and Deputy COP (DCOP) identified key evaluation stakeholders, reviewed the

EQs, and discussed evaluation use. It was agreed that USAID would provide comments in writing regarding the ET’s understanding of the EQs, especially EQ-4, and will also provide feedback regarding the list of stakeholders that the ET identified. The ET also requested several additional documents

(Annual Reports, sub-awardees, contracts, IPTT 2018 and 2019, the PEA, performance report, etc.) in order to proceed with preparing the Inception/Design Report. Categories of documents are fully referenced in Annex D.

The ET continued its review of URP Project documents, contacted Chemonics, the URP’s prime grantee, for additional documents and stakeholder contact information, and prepared the Inception and

Evaluation Design Report, which was submitted on December 23, 2020, and approved by USAID on

March 31, 2021.

PHASE TWO DATA COLLECTION

Following USAID/Haiti’s approval of the Inception and Evaluation Design Report, the ET launched the evaluation data collection phase on April 12, 2021 and collected data from April 12 to 30, 2021 (three weeks). The ET conducted the first days of data collection together to test and adjust the data collection protocols. Given the number of planned KIIs, the ET conducted several interviews with USAID and

13 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV

Chemonics together, but then divided into teams during the second and third weeks, to accommodate key respondents’ scheduling constraints and language preferences. The ET frontloaded the schedule with the most knowledgeable respondents, for example, USAID and Project (Chemonics, NSBA/CLSA, CIAT), who were well placed to provide context and highlight key issues that the ET could probe in later interviews with GOH officials, sub-awardees, municipalities, the private sector, academic institutions, and NGOs/CSOs. To accommodate potential technical challenges and rescheduling needs, the ET planned approximately three to four interviews per day, but the realities of respondents’ busy schedules meant these numbers ranged from one to seven interviews per day. The ET used non-interview time during the data collection phase to finalize notes, review FGD notes provided by CHASE, send follow-up emails to respondents, and create and update the emerging themes matrix throughout fieldwork.

The ET primarily used a qualitative evaluation design. The team extracted quantitative data from the

URP’s monitoring and evaluation (M&E) reports, including disaggregated data on URP-relevant indicators by partner institutions (when available) and gender to differentiate results. The ET triangulated the

Project M&E data with the qualitative data it collected through document review, KIIs and GIs with

Project stakeholders (sub-awardees, sub-watershed management committees, private sector, academic institutions, NGOs/CBOs, cooperatives, and other stakeholders), and FGDs with URP beneficiaries.

GUIDING PRINCIPLES

The ET conducted fieldwork with four guiding principles in mind:

1. Informed Consent: The ET administered informed consent scripts and asked for respondents’ verbal consent to ensure that respondents understood the purpose and voluntary nature of the

KIIs/FGDs, as well as the risks and mitigation measures associated with the COVID-19 pandemic in the case of FGDs.

2. Confidentiality: The confidentiality of key informants is a priority for ESS. The ET has produced a report that aggregates data and omits identifiers. Although the ET may use quotes, none of the individuals it interviewed are named in this report.

3. Gender Integration: USAID’s Automated Directive System (ADS) 205 requires gender integration into the evaluation design and implementation. ESS takes this guidance several steps further through SI’s Evaluation Quality, Use, and Impact (EQUI)® framework, which uses gender analysis frameworks to inform sampling strategies, gender-sensitive data collection protocols, sex-disaggregated data collection, and the consideration of gender and social dimensions in data analysis and reporting. A dedicated Gender Specialist, through sub-contractor EnCompass, supports the ET by reviewing deliverables at each stage and scoring against a Gender Scorecard, as well as contributing to team meetings throughout evaluation planning, data collection, and results debriefings. This scorecard synchronizes with EQUI’s quality assurance (QA) checkpoints to ensure that the ET integrates gender and social dimensions into each stage of the evaluation process, and as they relate to each evaluation topic. Furthermore, it paid particular attention to EQ-3, which specifically addresses integration of gender and youth into the URP. Data collection instruments included questions about the Project interventions’ effects on various gender groups, impacts on gender relations and equality, and intended and unintended impacts on women and men. Integration of gender into this evaluation includes review of the extent to which the activity (not just the evaluation) design, implementation, and MEL is consistent with the USAID Gender Equality and

Female Empowerment Policy and overall ADS 205 policy objectives and requirements for gender integration.

4. COVID-19 Safety Protocols: As highlighted above, because of COVID-19-related travel and meeting restrictions, the ET conducted all KIIs and GIs remotely. The FGD approach involved

USAID.GOV USAID/HAITI REFORESTATION PROJECT FINAL REPORT | 14

gathering farmers in a common location where they could sit socially distanced and participate in the

FGD via a tablet that was provided to them by the CHASE logistician. The reason for gathering farmers in one location for the focus groups is that farmers tend to lose focus when they are participating in FGDs at home and/or are not actually available at the agreed upon time, according to our recent experience. The ET believed it would not be possible to collect quality FGD data from farmers without gathering participants in a common location at a specified time. For this to happen, one team member needed to be in place to organize the location, provide the Internet and tablets, disinfect the space, and enforce distancing. This approach minimized physical interaction to the greatest extent possible, while simultaneously enhancing the possibilities of collecting quality data.

DATA COLLECTION METHODS

FGDs. The ET engaged the local firm, Collectif Haitien pour l’Avancement du Suivi et de l’Evaluation

(CHASE) to conduct FGDs with farmers throughout Haiti’s North and North-East departments. The ET conducted 20 FGDs in the five different sub-watershed areas that the URP is supporting. In each sub-watershed area, four FGDs were organized. The ET fielded two teams in order to carry out two FGDs simultaneously. This allowed the team to organize and complete all 20 FGDs in three weeks. The FGDs included women’s groups, youth associations, sub-watershed committee members, and farmers that benefitted from the Project. Each FGD included between five and fifteen participants. To maximize participation, the facilitator conducted the discussion in Creole, and called upon individual participants in the group to promote their active participation. Information gathered from FGDs informed the ET’s answers to all four EQs.

TABLE 1. LIST OF FGDS CONDUCTED

SUB-

WATERSHED

ASSOCIATION/G

ROUP

MALES FEMALES ASSOCIATION

ACTIVITIES

Limonade APPWOLIM/ Asosyasyon

Pwodikte Let nan Limonad/ Association d’eleveurs

7 1 Agroforestry, livestock and dairy

Ouanaminthe GEDWA/Association d’Eleveurs pour le Développement de Ouanaminthe

4 3 Livestock

Perches Bénéficiaires 3 7 Agroforestry, Livestock

Terrier Rouge KOET

Koperativ Elvè Terrier Rouge

8 7 Livestock

Trou du

Nord/Roche

Plate

AFDRP

Assosyasyon Fanm pou

Devlopman Roche Plate

0 12 Livestock

Ferrier AAF/ Association des Apiculteurs de Ferrier

9 5 Agroforestry, Bee-keeping

15 | USAID/HAITI REFORESTATION PROJECT FINAL REPORT USAID.GOV

Caracol FoProBiM/ APDK Asosyasyon Péchè pou Devlopman Karakòl

8 1 Mangrove Reforestation, Bee-keeping

Fort-Liberte CACOPA 4 3 Agroforestry, Bee-keeping

Madras APM/ Asosyasyon Pechè Madras

7 2 Mangrove Reforestation

Limonade OPPBL

Òganizasyon Pechè pou

Pwoteksyon Bòdmè Limonade

5 1 Mangrove Reforestation

Grande Rivière du Nord

CAJBC

Coopérative Agricole Jean Baptiste Chavannes

6 2 Agroforestry, Reforestation and Cacao

Quartier Morin OTG /Oganizasyon ti Plantè

Gran Pré

8 1 Agroforestry

Bois de Lance/ Limonade

MPDB

Mouvman Pwogresis pou

Devlopman Bois de Lance

6 2 Reforestation

Plaine du Nord FECCANO 6 2 Agroforestry, Cacao

Limonade PLOMB 8 1

Agroforestry

Vallières KODV

Komite Oganizasyon pou Devlopman Valyè

6 2 Reforestation

Vallières MPFV

Mouvman Pwogresis Fanm

Valyè

0 9 Reforestation

Vallieres ADTP 4 2 Agroforestry

Jaczil AFVMJ / Asosyasyon Fanm Vanyan Marais Salant Jaczil

0 5 Agroforestry

Ouanaminthe OTDO (Organisation des Travailleurs pour le Développement de Ouanaminthe)

1 5 Agroforestry, Livestock

Key Informant and Group Interviews. The ET conducted 28 KIIs and 5 GIs (with 19 participants) with representatives of key stakeholder groups, including USAID, grantees, and sub-awardees, GOH, sub-watershed committees, private sector, academic institutions, CSOs, and farmers’ associations. The

ET selected KII and GI participants using purposeful sampling techniques, supplemented by snowball sampling. Findings from KIIs and GIs contributed to the ET’s responses to all four EQs. Despite incorporating feedback from the Gender Scorecard to ensure sampling would be gender-inclusive within

USAID.GOV USAID/HAITI REFORESTATION PROJECT FINAL REPORT | 16

purposeful sampling, in the end the ET was not able to achieve a gender balance among KIIs and GIs; less than 25 percent of KII and GI respondents were women. In many cases, this was due to gender imbalances within the institutions, particularly among leadership positions.

The main stakeholder categories the ET interviewed are outlined in Table 2.

TABLE 2. STAKEHOLDERS INTERVIEWED DURING FIELDWORK

STAKEHOLDER

CATEGORY

ORGANIZATIONS REPRESENTED # MALES # FEMALES

USAID/Haiti AOR, Environmental Officer 1 1

Prime and Sub-primes

Chemonics, NCBA CLUSA/CIAT 5 2

Sub-Awardees Saint Barnabas Agriculture Center (CASB); APAPANNE

Dosmond Germplasm Center; FoProBiM; Kiskeya Vèt; Institution Univers, FECCANO, Village Planète, AAF, ODL, RECOCARNO, and MSPJ

10 2

Government Directions Départementales du Nord of the Ministries of Environment and of Agriculture

3 0

Sub-Watershed Management Committees

Joli-Trou/Cormier and Bahon (both in Grande Riviere du Nord watershed), Perches (Marion watershed), Gens de Nantes/Savane Longue (Jassa watershed), and Capotille (border area – Massacre watershed)

15 4

Private Sector Antillean Canning S.A. (ANCASA), One Tree Planted 1 1

Academic/Training Institutions

Centre de Formation en Aménagement Intégré des Mornes (CFAIM) and Université Henry Christophe de Limonade

2 0

Total 37 10

USAID personnel KIIs. The ET interviewed two USAID personnel: the USAID Activity Agreement

Officers’ Representative (AOR), who was responsible for overall Project oversight and coordination, and the Environment Officer, who provides technical oversight for several environmentally focused

USAID projects in Haiti, including the URP.

URP personnel KIIs. The ET interviewed key prime and sub-prime Project personnel, including the

COP, DCOP, and…