J.1c_SOO Annex 3_Lebanon-LIVCD Final Evaluation .pdf

November 2018

This publication was produced at the request of the United States Agency for International Development.

It was prepared by independently by Matteo Borzoni (Team Leader) with the support of Joanna Khater from Social Impact, Inc.

Note: Limited redactions have been made to this version of the report in accordance with the principled exceptions to the presumption in favor of openness established in OMB Bulletin 12-01, “Guidance on

Collection of U.S. Foreign Assistance Data.”

SENSITIVE BUT UNCLASSIFIED

LEBANON INDUSTRY VALUE

CHAIN DEVELOPMENT (LIVCD)

FINAL PERFORMANCE EVALUATION

FINAL REPORT

Performance Management and Support Program for Lebanon (PMSPL II)

SENSITIVE BUT UNCLASSIFIED

Lebanon Industry Value Chain Development

(LIVCD)

Final Performance Evaluation

Cover Photo: Workers sorting apples at the cold storage facility of Liban Village, Lebanon (Matteo Borzoni, Evaluation Team Leader/ Social

Impact)

Performance Management and Support Program for Lebanon (PMSPL II)

November 2018

This document was submitted by Social Impact, Inc., to the United States Agency for International

Development under USAID Contract/Order No. AID-268-C-15-00001.

This document is available printed or online. Documents are also made available through the Development

Experience Clearinghouse (http://dec.usaid.gov). Additional information is available from:

Social Impact, Inc.

2300 Clarendon Boulevard

Arlington, VA 22201

Tel: (703) 465-1884

Fax: (703) 465-1888 info@socialimpact.com

DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government.

mailto:info@socialimpact.com ii | LIVCD FINAL EVALUATION REPORT USAID/LEBANON

NOVEMBER 2018

SENSITIVE BUT UNCLASSIFIED

ABSTRACT

Implemented between September 2012 and January 2019, the USAID-funded Lebanon Industry Value

Chain Development (LIVCD) project sought to develop inclusive and competitive value chains in rural areas of the country to help increase the income of the population there, including micro, small, and medium enterprises. The evaluation examines the project’s relevance, efficiency and cost-effectiveness, effectiveness, sustainability, and additionality. It provides conclusions and recommendations to inform future activities and follow-on actions. The methods employed included a document review, 64 key-informant interviews, 28 focus group discussions (with 221 individuals), and direct observations.

LIVCD’s performance varied according to the targeted value chain. Interventions with avocado and olive farmers were the most cost-effective, while those with apple and cherry farmers were the least cost-effective. Significant contextual challenges hindered LIVCD’s performance. However, the evaluation found evidence of improved incomes and competitiveness for avocado, olive, and grape farmers in addition to processed-food companies. There was limited or no income improvement for LIVCD-supported apple and cherry farmers. The total value of loans raised was considerable. However, access to finance improved for 102 beneficiaries only and was rather the result of investments leveraged by grants and technical assistance than that of activities aimed at promoting access to finance. Additionality was evident when new technologies or techniques were introduced. The project design included aspects related to ensuring sustainability. However, without further improvements in the quality of the produce, the interventions’ sustainability is at risk, especially in the apple value chain.

USAID/LEBANON LIVCD FINAL EVALUATION REPORT | iii

SENSITIVE BUT UNCLASSIFIED

Table of Contents

ABSTRACT .................................................................................................................................... i

ACRONYMS .................................................................................................................................. v

EXECUTIVE SUMMARY

I. Introduction

II. The Development Problem and USAID’s Response

The Development Problem

USAID’s Response

III. Evaluation Purpose, Users, and Questions

Evaluation Purpose and Users

Evaluation Questions

EVALUATION TEAM AND SUPPORT STRUCTURE

IV. Evaluation Methodology

Data Collection Methods

Document Review

Key Informant Interviews

Focus Group Discussions (FGDs)

Observations

Data Analysis

Data Analysis Processes

Data Triangulation

Data Analysis Methods

Biases and Other Limitations

V. Findings and Conclusions

Evaluation Question 1

Findings

Conclusions

Evaluation Question 2

Findings

Conclusions

Evaluation Question 3

Findings

Conclusions

Evaluation Question 4 iv | LIVCD FINAL EVALUATION REPORT USAID/LEBANON

SENSITIVE BUT UNCLASSIFIED

Findings

Conclusions

Evaluation Question 5

Findings

Conclusions

VI. Recommendations

VII. Annexes

Annex A: Evaluation Statement of Work

Annex B: Data Collection Protocols

Key Informant Interviews

Focus Group Discussions

Annex C: Documents Reviewed

Annex D: Evaluation Respondents

Key Informants

FGD Respondents

Annex F: Evaluation Design Matrix

FIGURES AND TABLES

Figure 1: Number of Beneficiaries by Value Chain and by Sex

Table 1: Key Informants by Type and Gender

Table 2: FGD Groups by Value Chains

Table 3: Costs and Benefits by Value Chain

Table 4: Grant Budget by Value Chain

Table 5: Loans Awarded

Table 6: Distribution of Grants by Processing Time

Table 7: Number of Grants per Year file:///C:/Users/bkimp/Documents/LIVCD%20Final%20Evaluation%20Draft%20Report%2020190104bk.docx%23_Toc535177566

USAID/LEBANON LIVCD FINAL EVALUATION REPORT | v

SENSITIVE BUT UNCLASSIFIED

ACRONYMS

AUB American University of Beirut

BBS beneficiary-based survey

BCR benefit-cost ratio

CCIAT Chamber of Commerce Industry and Agriculture of Tripoli and North Lebanon

CCIAZ Chamber of Commerce, Industry and Agriculture of Zahle

CDCS Country Development Cooperation Strategy

CLA Collaboration, Learning and Adaptation

COP chief of party

DAI Development Alternatives, Inc.

DEC Development Experience Clearinghouse

DO development objective

DO2 second development objective

EoI expression of interest

EQ evaluation question

ESS electrostatic spraying system

ET evaluation team

FG focus group

FGD focus group discussion

FtF Feed the Future

FY fiscal year

GDP gross domestic product

GM gross margin

GNI gross national income

GOL Government of Lebanon

Ha hectare

ILO International Labor Organization

IP implementing partner

IPTT Indicators Performance Tracking Table

IR intermediate result

KI key informant

KII key informant interview

LIBNOR Lebanese Standards Institution

LIVCD Lebanon Industrial Value Chain Development

M&E monitoring and evaluation

MFI microfinance institution

MoA Ministry of Agriculture

MoT Ministry of Tourism

MSMEs micro-, small, and medium-size enterprises

MTE midterm evaluation

NGO nongovernmental organization

PMP performance management plan

PMSPL Performance Monitoring and Support Lebanon

RfP request for proposals

SI Social Impact

SoW scope of work

TA technical assistance

TL team leader vi | LIVCD FINAL EVALUATION REPORT USAID/LEBANON

SENSITIVE BUT UNCLASSIFIED

UNDP United Nations Development Program

USAID United States Agency for International Development

USEK Holy Spirit University of Kaslik

USG United States government

VC value chain

USAID/LEBANON LIVCD FINAL EVALUATION REPORT | 1

EXECUTIVE SUMMARY

INTRODUCTION

Executed by Development Alternatives, Inc. (DAI) from September 2012 to January 2019, the United

States Agency for International Development (USAID)’s Lebanon Industry Value Chain Development

(LIVCD) project is a $46.2-million activity that seeks to develop inclusive and competitive value chains to raise the incomes of the rural population in Lebanon, including micro-, small, and medium-size enterprises

(MSMEs).

Lebanon is an upper-middle-income country that has been experiencing a deep economic crisis since early

2012. Lebanese products and services suffer from a lack of competitiveness, largely due to the negative economic effects of 15 years of war and civil strife. Traditional markets and value chains have been disrupted, and private and public sector investment has stagnated. The impact on rural areas has been particularly acute as it has been exacerbated by an absence of public sector services such as agricultural extension. LIVCD aims “to develop functioning, competitive value chains to increase incomes of the rural population, including MSMEs.” To achieve its objective, LIVCD has collaborated with private actors within the selected value chains by providing technical assistance (TA), equipment, grants, and access to investment for value chain financing to overcome constraints to competitiveness and inclusiveness.

The project targeted eight value chains: i) pome fruits, ii) stone fruits (cherries and avocados), iii) table grapes, iv) olive oil, v) rural basket of products (including honey, eggs, pine nuts, and herbs), vi) processed foods, vii) rural tourism and handicrafts, and viii) floriculture. In 2014, floriculture and handicrafts were excluded from LIVCD’s portfolio.

This final evaluation documents LIVCD’s critical successes and shortcomings to provide the USAID

Mission in Lebanon, the U.S. government (USG), and the wider donor community with usable lessons learned for adaptation under the USAID Collaborating, Learning, and Adapting (CLA) framework. Through five evaluation questions (EQs), this evaluation examines the project’s relevance, efficiency and cost-effectiveness, effectiveness, and sustainability and additionality. It then provides conclusions and recommendations to inform future activities and follow-on actions. The key user is the economic growth team at USAID/Lebanon—specifically, the economic growth office, the program office, and others at the

Mission’s discretion. The evaluation’s results will be used by USAID/Lebanon in its annual portfolio review.

Using a theory-based, utilization-focused approach, the evaluation team (ET) employed a qualitative design consisting of the following data collection methods: document review, key informant interviews (KIIs), focus group discussions (FGDs), and observations. The ET also reviewed and used quantitative data

(indicators and survey data) collected by the project. The team followed a purposive sampling approach for KIIs and FGDs and conducted a total of 64 KIIs (with 71 individuals) and 28 FGDs (with 221 individuals).

Data triangulation was used to cross-check the results using several analysis methods, including content, contribution, comparative, and gender analysis, to provide evidence for the evaluation’s findings and conclusions.

FINDINGS AND CONCLUSIONS

EQ 1: To what extent were the value chain approach and LIVCD’s objectives and performance indicators appropriate for meeting the Country Development Coordination

Strategy (CDCS) development objective (DO)?

The value chain (VC) approach allowed the project to address all stages required to bring an agricultural product to the final consumer, allowing the project’s scope to target a wide range of economic actors.

2 | LIVCD FINAL EVALUATION REPORT USAID/LEBANON

SENSITIVE BUT UNCLASSIFIED

The VC approach also allowed the project to design cost-reduction and quality-improvement interventions with the aim of creating a more competitive product in terms of cost and quality.

Although processed foods and rural tourism are not value chains, they were included in the project because of their high potential to target women, unlike the male-dominated agriculture sector. In addition, the processed foods sector had a high potential to reduce poverty through supporting companies that create jobs and strengthening interventions for women’s cooperatives.

CDCS Development Objective 2 (DO2) is “Inclusive economic growth enhanced.” Comparing LIVCD’s

DO, goal, and intermediate results (IRs) with those of the CDCS shows that the project’s DO is well aligned with the CDCS. However, of the five LIVCD DO performance indicators, the indicator formulated as “number of jobs impacted” is very generic and not informative because it does not reflect job creation.

In addition, although LIVCD activities aimed to strengthen business associations and policy advocacy— areas explicitly covered by the CDCS—these activities are not reflected by any of the project’s indicators.

In conclusion, the VC approach aligned with DO2 since it was appropriate to improve the competitiveness of Lebanese products and enhance economic growth. The VC approach also made the interventions more inclusive by enabling LIVCD to target a wide range of economic actors. By adding rural tourism and processed foods to its portfolio, the project also enabled a greater inclusiveness of women. The project’s performance indicators were overall in line with the CDCS; however, the lack of an indicator reflecting job creation prevented a full appreciation of the project’s inclusiveness and contribution to the CDCS

DO.

EQ 2: To what extent were the selected interventions cost-effective?

The ET calculated the benefit-cost ratio (BCR) using the LIVCD farmer survey’s results. The BCR was high for those interventions with avocado farmers (6.74) and olive farmers (3.01). For avocado, this was due to the training, which increased the yield. For olive oil, the high BCR was a result of the introduction of mechanical harvesters, which reduced harvesting costs. The BCR was low for cherry (0.11), mainly because of lower yields, and negative for apple (-0.46), mainly because of a lower sale price resulting from the devaluation of the Egyptian pound and limited export market opportunities apart from Egypt.

The project asked grantees to contribute to the cost of grants. The grantees’ contribution was higher in the avocado value chain (160 percent of the awarded amount) and in the grape value chain (180 percent), while it was lower for honey (63 percent), rural tourism (65 percent), and processed foods (82 percent).

However, the project TA inspired significant investments among beneficiary companies of the processed foods value chain. Also, the processed foods companies visited by the ET reported increases in both volumes produced and sales. For the cherry value chain, the grantees’ contribution was higher than the awarded amount; however, the incremental gross margin (GM) of cherry farmers was lower than the project’s direct cost.

Rural tourism value chain beneficiaries reported that their jobs were not impacted by trainings but rather by the number of tourists visiting the target areas. Tourist numbers rose in the targeted areas, thus increasing the incomes of guesthouse owners, restaurants, and guides. However, since many other non-

LIVCD-funded interventions targeted the same areas, it is questionable to attribute the increase of tourists in target areas to LIVCD efforts.

In conclusion, interventions in the avocado and olive oil value chains were more cost effective than those in the other agricultural value chains. Interventions in the cherry and apple value chains were not cost effective.

Grantee contributions in the avocado and grape value chains added significantly to the interventions’ cost-effectiveness. In the processed foods value chain, the grantee contribution was lower than the grant-awarded amount. However, the TA-inspired investments among some of the beneficiary companies

USAID/LEBANON LIVCD FINAL EVALUATION REPORT | 3

enlarged their production capacities. In addition, the fact that the beneficiary companies in the processed foods value chain increased sales and volume produced suggests that interventions in this value chain were also cost effective.

EQ 3: To what extent have LIVCD’s interventions resulted in meeting the objectives and key performance indicators (KPIs) of the initial and extended contracts? Were the selected interventions and entry points in each value chain strategic in leading to the intended results?

What internal and external factors contributed to the achievement (or nonachievement) of the project’s results?

All project targets were achieved, with some exceptions for the GM indicators. The LIVCD farmer survey showed that changes in the GM per hectare have reached the 10 percent target increase for avocado, grape, and olive oil. The GM per hectare rose for cherry and honey, but the 10 percent increase was not met. The GM per hectare fell for apples. The target for women’s participation was achieved (23 percent of all beneficiaries were women). The FGDs revealed that the location and time of all training sessions conducted were suitable for women. However, women’s participation was low in all six agricultural value chains because agriculture is male-dominated and production is perceived as too tough for women.

Interventions were devised to enhance farmers’ individual productivity in value chains. However, all categories of economic operators reported significant contextual challenges, which include: a small and fragmented land ownership structure, the farmers’ old age, the closure of land border crossings, the dubious quality of pesticides sold by input providers, the unregulated import of olive oil, and the Ministry of Agriculture (MoA)’s weak capacity to provide TA and the laboratory analyses needed for exports

(Tripoli’s Chamber of Commerce provides all the required laboratory analyses to export fruits, but the fruit exporters we visited were using only MoA’s lab). With these conditions, it is very “difficult to beat the system” with training, TA, and the introduction of modern equipment.

However, both the processed foods sector and the olive oil value chain showed some evidence of greater competitiveness and higher incomes. This was achieved by introducing technological innovations and developing market linkages for some processed food companies. Incomes in the avocado and grape value chains also rose. The FGDs and the LIVCD survey’s results for the GM suggest that apple value chain competitiveness did not improve, and that project benefits were lower for cherry value chain beneficiaries.

The rising number of tourists visiting the areas targeted by the rural tourism value chain interventions generated higher incomes there. However, except for one village, the attribution of tourists increase to the project is questionable.

The total value of loans raised by LIVCD beneficiaries is considerable ($18.8 million). However, the loans were awarded to a very small number of beneficiaries (102). The project developed feasibility studies for

27 beneficiaries, which resulted in 12 awarded loans. The fact that only one beneficiary company used the

LIVCD feasibility study to raise loans suggests that the loans awarded to companies (40 percent of the total value of loans) resulted more from the investments leveraged by grants and TA than from the activities aimed at promoting access to finance.

The combined use of TA and grants allowed the project to tailor interventions to the investment capacities of beneficiaries and to their risk aversion profile. The project partnered with cooperatives, private companies, and the Ministry of Tourism (MoT). Targeting cooperatives allowed LIVCD to reach a high number of farmers at a low cost. Targeting cooperatives was also strategic to establish service centers that deliver pruning services and manage mechanical harvesters, but not to improve access to output markets. The targeted private companies that provide agricultural production and post-harvest services were instrumental in developing the chosen value chains. MoT’s involvement was strategic to develop the

4 | LIVCD FINAL EVALUATION REPORT USAID/LEBANON

SENSITIVE BUT UNCLASSIFIED

National Strategy on Rural Tourism. However, the fact that the political consensus so created was not used to develop the needed regulation was a missed opportunity.

The beneficiary organizations of processed-foods interventions were also strategic to create employment opportunities for women (and some people with disabilities) and to provide a market outlet for minor quantities of low-quality grapes and apples.

EQ 4: To what extent can LIVCD’s interventions be considered sustainable?

The project design had several aspects related to ensuring sustainability, such as requesting the grantees to contribute to grant costs and an intervention strategy focused on the private sector.

For the apple value chain, private sector involvement occurred during the final phase when a new large factory was established in Bcharre through LIVCD support. However, the factory foresees using just 30 percent of its capacity in 2018–19 because there are not enough quality apples in the region. Other activities were implemented in the same area, such as supporting an apple farmers’ cooperative and funding a small apple juice factory and an agro-input seller that also provides TA services to farmers.

For apple farmers, continued access to TA is facilitated by the fact that Bcharre’s six municipalities are currently employing agronomists to provide technical advice to farmers. The municipality of Baskinta is also employing an agronomist to follow up with apple farmers. Specialized teams offering pruning services were formed following the training provided by LIVCD and the equipment delivered.

For avocado farmers, TA in the South is provided by the experts that partnered with LIVCD throughout the project’s life. In the North the expert who was employed by the grantee left Lebanon. The ET found contrasting evidence on TA opportunities for avocado farmers, but in Akkar (only) an EU-funded project is providing TA to the same LIVCD beneficiaries. FGDs with avocado farmers revealed that avocados can be easily sold. LIVCD has also confirmed the growing demand for avocados.

In the grape value chains, two of the three exporters supported by the project have agronomists to provide TA to farmers. They have also expressed their willingness to buy grapes from the farmers to whom they were linked once the new plantations increase production (in 2019). However, they also reported that they would not engage in a similar project again since following up with smallholders is too complicated and expensive. The third supported exporter is no longer in the grape business in Lebanon.

LIVCD provided TA to grape farmers either directly or through the personnel employed through grants awarded to CCIAZ. However, the CCIAZ grape expert left three months before the end of the grant

(given the short remaining period no replacement took place) and the EU grape project implemented through CCIAZ targets only part of the LIVCD grape beneficiaries.

In honey, the use of TA services supported by the project depends largely on the distance from TA providers. Whereas four FGDs with beekeepers reported they were using TA and production services, the focus group in Baalbek-Hermel reported to have never used any TA service. In addition, the artificial insemination center for queen bees supported by the project is contested by academics.

In rural tourism, the local project initiatives are expected to be maintained by the grantees. The increase in the number of tourists visiting the target rural areas suggests that the rise in tourism income motivates a continuation of the services delivered. An important activity promoted by the project was the development of the National Rural Tourism Strategy; however, the strategy is no longer on the policy agenda since the minister of tourism changed. The steering committee responsible for the implementation of the strategy is no longer active.

An unplanned effect of the project was the establishment of four start-up companies that, at the time of the evaluation, were providing TA services to beekeepers and avocado farmers. They also maintain the roasting equipment and optical sorters delivered to food processing companies or cooperatives.

In conclusion, the project design included aspects related to ensuring sustainability. For the apple value

USAID/LEBANON LIVCD FINAL EVALUATION REPORT | 5

chain, this also included concentrating interventions in a single region. However, without clear improvements in produce quality, the sustainability of interventions is at risk. Continued access to TA is provided by two of the three grape exporters for 30 grape farmers only, and CCIAZ’s capacity to provide

TA to grape farmers is limited. Access to TA is easier for those avocado farmers in the South, while ET found conflicting evidence of available TA opportunities for avocado farmers in Akkar. Interventions in the honey value chain resulted in some groups having easier access to TA and production services and others having none. For rural tourism, while local interventions have good prospects of sustainability, the development of the National Strategy did not prove to be sustainable.

EQ 5: Were the positive results spread across all beneficiaries or mainly to few ones only?

And to what extent did the interventions demonstrate additionality?

LIVCD survey results showed that four categories of beneficiary farmers—grape, avocado, olive, and honey producers—experienced benefits that can be attributed to the project, while two categories— apple and cherry farmers, which constitute 17 percent of the beneficiary farmers—did not. Beneficiaries of the interventions in the processed foods and rural tourism sectors were not included in the LIVCD farmer survey. Five beneficiary companies (out of six interviewed by the ET) of the processed foods interventions reported an increase in the number of employees, sales, and volumes produced as a result of LIVCD support. Four companies also reported an increase in the number of farmers from whom they were buying produce (one company had just installed the equipment funded by USAID).

The LIVCD impact study shows that incomes rose for the beneficiaries of 14 out of 18 rural tourism interventions. Four interventions did not deliver results, with no benefits accruing to beneficiaries.

The project reached the target for women’s participation, but women’s participation was low in all agricultural value chains. A gender analysis was not systematically included in the initial value chain assessment studies. Also, the fact that LIVCD’s staff could not recall the Resource Guide for Gender

Integration in Value Chains developed by PMPSL II for the Mission indicated that the resource guide was not used.

Interventions promoting avocado cultivation were additional because avocado was introduced in new areas and because farmers reported having learned new techniques. Grape farmers also reported that the

LIVCD contribution was essential to establish new plantations and install drip irrigation systems.

Beekeepers reported mixed considerations on the additionality of beehive distribution, with some reporting that no hives would have been bought without LIVCD’s support and others stating that hives are affordable and would have been bought without support.

Beneficiaries of the technological innovations introduced by LIVCD stated that their investments would not have been possible without this assistance. These investments included mechanical harvesters, cucumber optical sorters, tank fermenters for pickles, freekeh roasters, and a beeswax recycling center.

Mixed considerations were reported by the companies that received grants for post-harvest services, with two of them stating that without LIVCD’s assistance there would have been no (or very different) investments, and two others reporting that the investments were already planned but that LIVCD had accelerated the process. Replications occurred for the mechanical harvesters for olives, the cucumber optical sorters, and the freekeh roasters.

In conclusion, benefits were spread across all beneficiaries except for apple and cherry farmers, who are a minority of the beneficiary farmers. However, across the six agricultural value chains, more benefits accrued to men than to women because the percentage of male farmers is much higher than that of female farmers. This is mainly explained by the fact that agriculture in Lebanon is a male-dominated environment.

Interventions in the processed foods sector created jobs and increased the number of farmers supplying the companies and the quantity of raw material bought. The additionality of grants and TA was evident

6 | LIVCD FINAL EVALUATION REPORT USAID/LEBANON

SENSITIVE BUT UNCLASSIFIED

for interventions introducing technologies hitherto unknown to beneficiaries. Grants for avocado and grape farmers can be considered additional. Additionality was less evident for grants awarded to well-established companies and that did not introduce innovations. Also, hive distribution was not completely additional.

RECOMMENDATIONS

1. Continue using grants to introduce innovations among risk-adverse beneficiaries and provide TA to well-established companies.

2. Prioritize interventions aimed at developing the Bcharre apple value chain to make Bcharre a model to be showcased and replicated.

3. Consider providing support to develop regulations with the purpose of promoting an enabling environment for private sector development projects.

4. Continue targeting both private companies to improve access to international markets and cooperatives to establish service centers for input provision and TA.

5. In future private sector development agriculture-related activities, consider capturing the income generated, employment created, and changes in the enabling environment.

6. Seek assistance from external and independent experts on the relevance and sustainability of risky and contested grant proposals when dealing with interventions with high scientific content.

7. Consider further interventions in the cherry value chain.

8. Consider developing interventions to promote avocado exports.

9. Ensure that agriculture project contractors apply a gender integration strategy in all project aspects.

10. Consider ways to support the Ministry of Agriculture in key policy areas.

SENSITIVE BUT UNCLASSIFIED

I. INTRODUCTION

Executed by Development Alternatives, Inc. (DAI) from September 2012 to January 2019, the United

States Agency for International Development (USAID)/Lebanon Industry Value Chain Project (LIVCD) is a $46.2 million activity that sought to develop inclusive and competitive value chains to raise the incomes of the rural population, including micro-, small, and medium-size enterprises (MSMEs).

This report details the results of a final performance evaluation of LIVCD, conducted by Social Impact, Inc. (SI) for USAID/Lebanon as part of the Performance Monitoring and Support Lebanon (PMSPL II) project.

II. THE DEVELOPMENT PROBLEM AND USAID’S RESPONSE

THE DEVELOPMENT PROBLEM

The gross national income per capita in Lebanon in 2017 was $13,040,1 ranking Lebanon as an upper-middle-income country. Poverty in Lebanon was estimated at 27 percent as of 2011.2 According to a

2004–05 United Nations Development Program (UNDP) study—which continues to be used by policymakers to design poverty reduction policies and programs)3—more than 20 percent of households engaged in agriculture are below the poverty line.4 (In 2016, UNDP conducted a new rapid poverty assessment in Lebanon, but results are not yet available.)

Gross domestic product (GDP) growth was 2.8 percent in 2012 and constantly declined through 2015, reaching 0.8 percent. In 2016 and 2017, GDP growth was 2 percent.5 The contribution to the GDP of the value added produced by the agriculture sector was 3.7 percent in 2012, 4.1 percent in 2014, and 3.5 percent in 2016 and 2017. According to International Labor Organization (ILO) estimates, the percentage of employment in agriculture was 3.2 from 2014 to 2017.5

The 2010 LIVCD request for proposals (RfP) emphasizes that many Lebanese products and services suffer from a lack of competitiveness, due largely to the negative economic effects of 15 years of war and civil strife. Traditional markets and value chains have been disrupted, and private and public sector investment has stagnated. The impact on rural areas has been particularly acute as it has been exacerbated by an absence of public sector services such as agricultural extension. Many rural areas have as a result become increasingly marginalized and economically isolated.

When the project was designed, the LIVCD RfP identified the following gaps constraining the capacity of rural value chain actors to compete in international or even domestic markets: i) the lack of extension services; ii) the absence of municipal revenue to fund market infrastructure; iii) the limited access to credit in many rural areas; iv) the lack of market intelligence to inform decision making and preserve capital; v)

1 Current prices, WB Atlas method. [Online] Available from: http://databank.worldbank.org/data/home.

2 UNDP. (2016). Rapid poverty assessment for Lebanon in 2016. Press release. [Online] Available from:

http://www.lb.undp.org/content/lebanon/en/home/Response_to_the_Syrian_Crisis/successstories/Rapid-Poverty-

Assessment-in-Lebanon-for-2016.html 3 Nuktrety N. & Al Jamal S. (2016). Poverty Inequality and Social Protection in Lebanon. American University of

Beirut and Oxfam. [Online] Available from:

http://website.aub.edu.lb/ifi/publications/Documents/research_reports/20160426_poverty_inequality.pdf 4 UNDP (2008). Poverty, Growth, and Income Distribution in Lebanon. UNDP. [Online] Available from:

http://www.undp.org/content/dam/lebanon/docs/Poverty/Publications/Poverty,%20Growth%20and%20Income%20D istribution%20in%20Lebanon.pdf 5 World Bank. [Online] Available from: Available from: http://databank.worldbank.org/data/home.

8 | LIVCD FINAL EVALUATION REPORT USAID/LEBANON

SENSITIVE BUT UNCLASSIFIED

the lack of transparency in value chain market information and transactions; vi) the high production costs,

vii) the high transaction costs; viii) the general absence of appropriate post-harvest handling, causing large crop losses; and ix) the lack of agricultural product sorting, grading, packaging, and cold storage facilities in some areas.

Two types of agriculture are predominant in Lebanon: commercial agriculture, which uses modern production and post-harvest techniques for commodities destined for export and domestic markets, and family agriculture, which is devoted to preserving cultural and family heritage.

The agriculture sector is male dominated. The ILO employment estimates from 2016 show that4.2 percent of men were employed in the agriculture sector, while just 0.045 percent of women employment was in that sector. However, this gender gap explains the male dominance in the agriculture sector only to a very limited extent. In 2016, males over 15 constituted 71 percent of the labor force, whereas women over 15 constituted 23 percent.6 Moreover, the LIVCD midterm evaluation analysis showed that women are paid less than men in agriculture. For example, in harvesting work women’s daily wage is 65 percent lower than that of men.

Food exports represented 14.9 percent of total merchandise exports in 2010 and 29.9 percent in 2016.5

These figures indicate the increasing importance of the agro-food sector among exports.

Overall, agricultural production depends largely on the immigrant workforce (mainly Syrian), especially for casual labor. The midterm evaluation reports that in the LIVCD target crops, the average number of non-Lebanese workers during harvest operations per farm is more than double the number of Lebanese workers.

USAID’S RESPONSE

The LIVCD project’s RfP emphasizes that the economic growth the country has been witnessing since

2007 has benefited many sectors of the national economy. However, rural areas, which depend primarily on agriculture and, to some extent, tourism, did not benefit much. They continued to suffer from limited private sector investment and access to credit. The RfP also stresses that a lack of adequate value chain finance is cited in several reports as an important factor limiting famers’ ability to participate in high value chain production.

Lebanon has significant natural resources and advantages, including sizeable expanses of fertile soils, a multitude of micro-climates, the capacity to produce early season fruits and vegetables, a close proximity to Gulf and European markets, abundant natural landscapes and heritage, ecological diversity, a multitude of religious and historical sites, a liberal business environment, a large entrepreneurial population, and a large Lebanese diaspora.

Lebanon has a potential competitive advantage in fresh produce (fruits and vegetables) and agro-processing. The comparatively underdeveloped state of Lebanon’s ecological, historical, and religious assets suggests that additional opportunities to create rural wealth exist in rural tourism, including agro-tourism.

USAID’s LIVCD project was originally designed as a five-year project to respond to the above-mentioned sector problems. USAID signed the LIVCD contract (No. AID-268-C-12-00001) with DAI in September

2012 for an initial value of $41,682,272 until September 30, 2017. A contract modification was made on

June 23rd, 2017, extending LIVCD by 16 months until January 2019 and increasing the budget by $4.5 million, thus totaling $46,182,272.

6 ILO estimate. [Online] Available from: http://databank.worldbank.org/data/home.

USAID/LEBANON LIVCD FINAL EVALUATION REPORT | 9

The LIVCD project was designed to partner with the private sector and to apply the value chain approach to enhance inclusive economic growth, increase private sector competitiveness, and improve the incomes of the rural population.

No theory of change or development hypothesis was used to design the LIVCD project. A development hypothesis was later added during the development of the LIVCD monitoring and evaluation plan. The hypothesis states that “If actors, including micro, small, and medium enterprises (MSMEs) in the target value chains increase investment into their businesses to increase productivity, make new business linkages, and identify and enter new markets, then the value chain will be more competitive and MSME incomes will rise.”

The project’s purpose is “to develop functioning and competitive value chains to increase the incomes of the rural population, including MSMEs.”

A functioning value chain is defined in the contract as a competitive and inclusive value chain.

Competitiveness is thus measured by the increase in sales and productivity and the improvement of product quality, and inclusiveness by the number of value chain participants, including MSMEs, farmers, and other organizations receiving assistance.

To achieve its objective, LIVCD collaborates with actors within the selected value chains by providing technical assistance (TA), equipment, grants, and access to investment for value chain financing to overcome constraints to competitiveness and inclusiveness. The LIVCD activities are designed to achieve the following four LIVCD intermediate results (IRs):

• IR1: Increased access to markets in target value chains.

• IR2: Increased business linkages in target value chains.

• IR3: Increased productivity in target value chains.

• IR4: Constraints to lending and investment reduced in target value chains.

According to the contract modification signed in 2017, it is expected that by the end of the project the following results will be achieved:

• A minimum of seven functioning competitive value chains.

• At least 17,000 MSMEs, including farmers and other organizations receiving business development services.

• At least 12,500 small and medium-size commercial growers benefiting from the dissemination of improved production and post-harvest technologies, and MSMEs who have applied improved technologies or management practices.

• At least 1,250 MSMEs, including farmers, reached by LIVCD to apply for value chain finance.

• At least $30 million of new private sector investment leveraged by LIVCD, including investment by farmers and other value chain actors.

• An increase of at least 5,000 businesses, farmers, or micro-enterprises benefiting from horizontal and vertical linkages.

• At least 150 new strengthened export markets, niche markets, or distribution channels for selected value chain products.

The LIVCD contract scope of work (SoW) specifies that the contractor is expected to undertake tasks under the following three components:

1) Conduct a complete assessment of the prospects for rural value chains in Lebanon based on secondary and primary market research, analysis of the current environment for value chains in

10 | LIVCD FINAL EVALUATION REPORT USAID/LEBANON

SENSITIVE BUT UNCLASSIFIED

Lebanon, and a reconciliation of market intelligence with the capacity of Lebanon’s rural sector to select the value chains for LIVCD intervention.

2) Undertake value chain upgrading activities to increase the competitiveness of selected Lebanese value chains that have the potential to raise incomes of the rural population, supply key markets, and increase export growth and/or the number of rural tourists.

3) Assist in accessing value chain finance for producers and other value chain actors in the rural economy.

In May 2013, eight value chains were approved for inclusion in the project: i) pome fruits, ii) stone fruits

(cherries and avocados), iii) grapes, iv) olive oil, v) rural basket of products (including honey, eggs, pine nuts, and herbs), vi) processed foods, vii) rural tourism and handicrafts, and viii) floriculture. In May 2014

LIVCD, with the approval of USAID, eliminated floriculture from LIVCD’s portfolio.

The project was implemented in all governorates of Lebanon.

III. EVALUATION PURPOSE, USERS, AND QUESTIONS

EVALUATION PURPOSE AND USERS

This final evaluation documents LIVCD’s critical successes and shortcomings to provide the Mission, the

U.S. government (USG), and the wider donor community with usable lessons learned for adaptation under the United States Agency for International Development (USAID) Collaborating, Learning, and Adapting

(CLA) framework. The evaluation examines the project’s relevance, efficiency and cost-effectiveness, effectiveness, and sustainability and additionality. It then provides conclusions and recommendations to inform future activities and follow-on actions.

The key user is the economic growth team at USAID/Lebanon—specifically, the economic growth office, the program office, and others at the Mission’s discretion. The evaluation results will be used by

USAID/Lebanon in its annual portfolio review and by the LIVCD implementer, DAI, to inform future program implementation. The Government of Lebanon and key donors might also use the evaluation recommendations to design new interventions in the sector.

EVALUATION QUESTIONS

The LIVCD final evaluation scope of work contained eight illustrative evaluation questions (EQs), exceeding the recommendations provided by the relevant USAID How-To Note.7 Moreover, some of the

EQs were not researchable because they asked what the result could be if an alternative approach were implemented, with no actual implementation of the envisaged alternative approach.8 The evaluation team worked with USAID to validate and improve the EQs. In agreement with USAID/Lebanon, the original set of questions was reformulated into the following five questions:

7 USAID. (2016). How-To Note: Evaluation Statement of Work. Program Cycle Guidance. [Online] Available from:

https://usaidlearninglab.org/library/how-note-evaluation-statement-work 8 An example is “Was the project able to achieve more value for money than alternative approaches (achieve the same results at a lower cost)? If yes, what would have been the alternatives?”

USAID/LEBANON LIVCD FINAL EVALUATION REPORT | 11

Relevance

1. To what extent were the value chain approach and LIVCD’s objectives and performance indicators appropriate for meeting the Country Development Cooperation Strategy (CDCS) development objective?

Efficiency and cost-effectiveness

2. To what extent were the selected interventions cost effective?

Effectiveness

3. To what extent have LIVCD’s interventions resulted in meeting the objectives and key performance indicators (KPIs) of the initial and extended contract?

a. Were the selected interventions and entry points in each value chain strategic in leading to the intended results?

b. What internal and external factors contributed to the achievement (or nonachievement) of the project’s results?

Sustainability and additionality

4. To what extent can LIVCD’s interventions be considered sustainable?

5. Were the positive results spread across all beneficiaries or only a few? And to what extent did the interventions demonstrate additionality?9

EVALUATION TEAM AND SUPPORT STRUCTURE

SI’s Performance Monitoring and Support Lebanon (PMSPL) II fielded a two-person evaluation team possessing a complementary mix of evaluation, value chain analysis, agricultural, and methodological knowledge. The team leader (TL), Matteo Borzoni, is an expert in value chain development and agro-economics. He has a strong interdisciplinary background and 15 years of experience as a consultant, researcher, and project manager for a wide range of donors, including USAID. Since 2002, Dr. Borzoni has been involved with implementations and evaluations of value chain development projects in Latin

America, the Middle East, Africa, and Asia. Joanna Khater, the local evaluation expert, has conducted evaluations and monitoring assignments for USAID since 2013. Both the TL and the local expert have participated in the midterm evaluation. Harvey Herr, PMSPL II’s chief of party (COP); Carine Khoury, PMSPL II’s deputy COP; and Najwa Andraos, PSMPL II’s gender specialist, provided advice and ad hoc inputs throughout the evaluation.

9 USAID requires grants and other interventions, especially to private sector entities, to demonstrate "additionality," or evidence that similar outcomes targeted by the grant or intervention would not have been realized in the absence of the USAID intervention. Simply citing the likely benefits and outcomes attributed to the grant/intervention is not sufficient justification, as it does not meet the additionality requirement. Evidence of additionality is normally provided by offering evidence that the purposes or proceeds of the grant/intervention: 1) could not, or likely would not, have been financed or purchased by the proposed beneficiary without USAID support; 2) serve a risk-reducing purpose, catalyzing the beneficiary to make an investment in a mostly unfamiliar technology with locally unproven benefits; 3) produce a demonstration effect, meaning that the success of the grant will induce significant numbers of others to copy the intervention, thus magnifying its impact; 4) allows the beneficiary to access finance that they otherwise could not have obtained; 5) crowds in outside resources/investment by offering a missing component to a transaction that induces other parties to participate and dedicate resources to the activity; 6) adds a noncommercial but important development component to a larger activity.

12 | LIVCD FINAL EVALUATION REPORT USAID/LEBANON

SENSITIVE BUT UNCLASSIFIED

PMSPL II subcontracted InfoPro, a Lebanese research firm, to conduct focus group discussions (FGDs) and provide support in qualitative analysis. The firm consists of researchers with experience using focus groups in different sectors. Each focus group was facilitated by a moderator, who was supported by a note taker. InfoPro also contracted Dr. Ali Chalak, an associate professor from the Faculty of Agriculture at the American University of Beirut (AUB), to analyze the FGD results.

IV. EVALUATION METHODOLOGY

The evaluation team took a theory-based, utilization-focused approach. Theory-based evaluation focuses on providing an in-depth analysis of a program’s underlying logic and causal linkages. It recognizes that a multitude of factors and interactions influence a project’s impact and looks to identify causal factors judged to be most critical to a project’s overall success. A utilization-focused approach is based on the principle that evaluations should be judged on their usefulness to their intended users; therefore, they should be planned and conducted in ways that enhance the likely utilization of both the findings and the process itself to inform decisions.

This final evaluation employed a qualitative design consisting of the following data collection methods:

document review, key informant interviews (KIIs), FGDs, and observations, described below. A structured survey (with quantitative data) was used for the midterm evaluation. In addition, LIVCD conducted a large survey (with a sample size of 907 farmers) to estimate the gross margins of beneficiaries. The final evaluation used the survey conducted at the time of the midterm evaluation and the results data made available by the project (including the LIVCD survey conducted to estimate gross margins). In this way, the findings obtained by the qualitative design of this evaluation were complemented by a quantitative analysis of already available datasets and project indicators.

Interventions were assessed in all value chains except rural baskets and floriculture. The latter was eliminated from LIVCD’s portfolio at the beginning of the project. The rural baskets value chain was not assessed upon request from USAID, given that interventions under this category were mainly about livelihood diversification rather than development of competitive agricultural value chains.

DATA COLLECTION METHODS

Document Review

The ET benefited from a relatively rich project document set. Before beginning fieldwork, the team reviewed the evaluation’s SoW; LIVCD’s project description, results framework, and monitoring and evaluation (M&E) plan; project studies (market studies, initial value chain assessments, impact assessments); annual and quarterly reports; and project contract documents and workplans. These documents provided a useful overview of the diversity of project activities. In addition, the ET used the farmer survey conducted by LIVCD (which compares the 2017–2018 and 2013–2014 agricultural seasons) to analyze cost-effectiveness (EQ2) and inform EQ3 and EQ5.

The ET also reviewed a prioritized list of non-project and secondary documents (Annex C), mainly about projects in Lebanon focused on agriculture, agricultural value chains, agribusiness, and…