Attachment J.13_ PROFIT+ Final Evaluation Report - Amendment I.pdf

December 2019 This report was produced at the request of the United States Agency for International Development (USAID). It was prepared independently by Integra LLC under the Learning, Evaluation, and Analysis project (LEAP III).

EX-POST EVALUATION OF

USAID/ZAMBIA’S PRODUCTION,

FINANCE, AND IMPROVED TECHNOLOGY

PLUS (PROFIT+)

FINAL EVALUATION REPORT

EX-POST EVALUATION OF USAID/ZAMBIA’S

PRODUCTION, FINANCE, AND IMPROVED

TECHNOLOGY PLUS (PROFIT+)

Contract Title: LEAP III: Learning, Evaluation and Analysis Project

Contract Number: GS-10F-083CA / 7200AA18M0004

Activity Number: LEAP III 2019 – 1009.1027

Submitted: December 13, 2019

Contractor: Integra Government Services International LLC 1100 Vermont Avenue NW, Suite 750 Washington, DC 20005

USAID Office: USAID/Zambia

COR: Yoon Lee, ylee@usaid.gov

DISCLAIMER

This report was produced at the request of the United States Agency for International Development (USAID) and was prepared independently by Integra LLC, under the Learning, Evaluation, and Analysis project (LEAP III).

TABLE OF CONTENTS

ACRONYM LIST 4

EXECUTIVE SUMMARY 6

I. INTRODUCTION 9

II. EVALUATION METHODOLOGY 14

III. EVALUATION FINDINGS 18

IV. CONCLUSIONS 43

V. REFERENCES 49

ANNEX 1: LITERATURE REVIEW 51

ANNEX 2. CAD SURVEY QUESTIONNAIRE 58

ANNEX 3. FOCUS CROP TRADE PERFORMANCE 64

ANNEX 4. DATA TABLES FROM THE CAD SURVEY 70

ACRONYM LIST

APS Annual Program Statement

CAD Community Agro-Dealer/Dealership

CFS Crop Forecast Survey

CFU Conservation Farming Unit

COMTRADE Commodity Trade Statistics Database

COR Contracting Officer Representative

CRS Catholic Relief Services

DACO District Agriculture Coordinator

DCOP Deputy Chief of Party

DHF Demonstration-Host Farmer

FGD Focus Group Discussion

FISP Farmer Input Support Program

FtF Feed the Future

GRZ Government of the Republic of Zambia

HA Hectare

ICT Information Communication Technology

IIP Innovation, Investment, and Partnership

KII Key Informant Interviews

LEAP III Learning, Evaluation, and Analysis III Project

M&E Monitoring and Evaluation

MoA Ministry of Agriculture

NGOs Non-Governmental Organizations

PACRA Patents and Companies Registration Agency

PC Producer Company

PROFIT+ Production, Finance, and Improved Technology Plus

R&D Research and Development

SILCs Saving and Internal Lending Communities

SMFM Sell More for More

SNAP Second National Agricultural Policy

SOW Scope of Work

USAID United States Agency for International Development

WRS Warehouse Receipt System

ZAMACE Zambia Commodity Exchange

ZARI Zambia Agriculture Research Institute

ZEMA Zambia Environmental Management Agency

ZOI Zone of Influence

ZRA Zambia Revenue Authority

6NDP Sixth National Development Plan

7NDP Seventh National Development Plan

EXECUTIVE SUMMARY

BACKGROUND AND METHODOLOGY

The United States Agency for International Development (USAID)/Zambia requested the Learning, Evaluation, and Analysis III project (LEAP III)1 to conduct an ex-post evaluation of the Production, Finance, and Improved Technology Plus (PROFIT+) project, which was implemented from 2012 to 2017. The objective of PROFIT+ was to improve productivity, expand trade, and increase investments by developing functional market systems in rural areas. During the first half of PROFIT+, the project focused primarily on the Demonstration-Host Farmer (DHF) model, where lead farmers promoted awareness about improved agricultural practices. Mid-way through the project, PROFIT+ adjusted the project’s focus to training and supporting DHFs to establish Community Agro-Dealerships (CADs). In addition to shifts in project activities, PROFIT+ also changed management and staffing considerably throughout its lifecycle.

The objectives of the evaluation are to identify: 1) key successes and challenges of PROFIT+ and critical processes/activities that enabled or contributed to these; 2) the merits and shortcomings of PROFIT+, as it directly relates to the CAD model and the Innovation, Investment, and Partnership (IIP) Fund; and 3) why activities have or have not continued beyond the life of project. USAID/Zambia provided five evaluation questions, which specifically focuses on sustainability and lessons learned of the project. The evaluation team utilized a mixed methods approach, combining a mix of qualitative key informant interviews (KIIs), focus group discussions (FGDs), desk research and a quantitative survey to assess the successes, challenges, and sustainability of the CADs and the IIP Fund established under PROFIT+. It should be noted that, at the direction of USAID/Zambia, this evaluation is not conducted as a traditional performance evaluation to assess whether the overall targets of the project were met, but rather an assessment of the effectiveness and sustainability of the project.

KEY FINDINGS

Based on the evaluation results and consultations with USAID/Zambia, the team organized the key findings under the two main themes: 1) Sustainability and 2) Lessons Learned.

SUSTAINABILITY

Key Successes

• Agriculture and business trainings were immensely popular and are still being used today. Almost all CADs, whether they were operating a CAD business or not, praised trainings as the most valuable aspect from PROFIT+. Many still use improved agricultural techniques and continue to share advice with neighbors and members of their community.

• Active CADs are doing better now than they were when PROFIT+ ended. CADs currently operating an agro-dealership, have more clients and sell more input than they did when PROFIT+ ended.

1 Implemented by Integra Government Services International LLC (‘Integra’) – www.integrallc.com.

• Active CADs maintain and continue to value linkages with input suppliers fostered under PROFIT+. CADs are primarily selling seeds and take care to maintain their relationships with suppliers.

• Women CADs are more empowered in their business, household, and community than before they started the CAD business. The success and failure rate between male and female CADs are comparable. However, women CADs have gained greater agency, voice, and leadership in their home and in their community since they started operating their agro-dealership.

• Saving and internal lending communities (SILCs) formed under PROFIT+ are still in operation. SILCs, dominated by women, continue to serve as an important customer base for CADs.

Key Challenges

• Only 42 percent of men and women that were trained to become a CAD are currently operating today. One quarter of trained CADs opened an agro-dealership but later closed it. Most did not have the financial means to establish a shop and were ill equipped to launch and run the business.

• Producer companies (PCs) are struggling or have failed completely. PCs were formed too late in the project and CADs forming the PCs were too unexperienced to launch a new business venture.

• Investment under the IIP Fund have, in individual cases, spurred change, but as a whole, resulted in uneven impact. The IIP funds were used across a vast spectrum of activities, but few of them truly bought down the risk of investing in the launch or growth of agro-businesses.

LESSONS LEARNED

Key Successes

• Participants are still using knowledge from PROFIT+ trainings. Trainings provided under this program proved to be extremely successful under PROFIT+. 97 percent of CADs found the business training to be valuable, and 88 percent found the training on how to operate the business as the most valuable support they received from PROFIT+. CADs are still using the agricultural techniques and conservation farming approaches taught under PROFIT+ to improve the yield on their farms.

• Focus on gender inclusion provided PROFIT+ with increased women participation and had positive outcomes on women. PROFIT+ made an active effort to include women as participants in all project activities, ensuring that all activities were implemented in an environment conducive to women. Throughout the evaluation, women reported feeling more empowered after working with PROFIT+, with greater agency, voice, and leadership amongst households.

Key Challenges

• Changes under PROFIT+ had lasting consequences on overall project performance and impact. PROFIT+ not only had management and staffing changes but shifts in project activities throughout the years. As a result, activities introduced or scaled up towards the end of the project received limited support and impact after PROFIT+ closed.

• Project close-out also came as a surprise to many project participants. Many project participants were completely unaware and unprepared that PROFIT+ was coming to an end. A better communicated exit strategy could have prepared project participants for greater resilience beyond the life of the project.

• Unclear processes in selecting CADs contributed to a high rate of non-operating agro-dealerships. Many DHFs were selected to participate in the CAD training. However, the majority had limited financial resources and business experience, which posed a considerable challenge for establishing a CAD.

• A lack of focus and purpose for the IIP Fund made it difficult to assess its impact. The IIP Fund provided grants with unclear purpose, selection criteria and timeline. Beyond individual cases, it is hard to see how the activities funded under the IIP Fund provide opportunities for agro-business growth, stimulate private sector investment, and expand Zambia’s agriculture sector.

I. INTRODUCTION

The United States Agency for International Development (USAID)/Zambia engaged Integra Government Services International through the Learning, Evaluation, and Analysis III project (LEAP III) to conduct an ex-post evaluation of the Production, Finance, and Improved Technology Plus (PROFIT+) project (2012- 2017), a five-year contract managed by ACDI/VOCA.

The objectives of the evaluation of PROFIT+ are threefold:

• Objective 1: Identify key successes and challenges of PROFIT+ and the critical processes/activities that enabled or contributed to them;

• Objective 2: Identify the merits and shortcomings of PROFIT+ Community Agro-Dealers (CAD) model; and

• Objective 3: Identify why activities have continued or not continued beyond the life of the project.

PROFIT+, a $24 million contract between USAID/Zambia and ACDI/VOCA, was Zambia’s Feed the Future (FtF) flagship project. The goal of PROFIT+ was to increase food security and decrease hunger through agriculture-led growth and inclusive market access by smallholder farmers. This was to be achieved through improved productivity, expanded agricultural trade, and increased agricultural investment. The project aimed to achieve a 30 percent increase in agricultural productivity and a $125 million increase in value of agricultural sales, resulting in an increase of incomes for 200,000 smallholder farmers, processors, and other value chain actors in four of the nine districts of the Eastern Province by 2017.

To this end, PROFIT+ identified and supported lead farmers to launch and operate small enterprises, known as CADs. The CAD activity was designed as a market approach to fill the often missing “last mile” between farmers, input suppliers, and output buyers. By establishing linkages for CADs with input suppliers and output buyers, PROFIT+ sought to address smallholders’ limited access to input and output markets, and to foment entrepreneurial activities and employment opportunities in rural areas. In addition, PROFIT+ established the Innovation, Investment, and Partnership (IIP) Fund, a grant facility to foster innovation, leverage resources, and to address market linkage constraints by buying down the risk of capital investments. To this end, the IIP Fund invested in input companies to develop their rural presence and linkages with the CADs in out-grower schemes also linked to CADs; and in agricultural processing facilities to improve, diversify and increase processing capacity. Across these activities, PROFIT+ sought to increase women’s participation and empowerment by engaging women as CADs and investing in women’s entrepreneurship.

PROFIT+ operated in the Eastern Province (Petauke, Katete, Chipata, and Lundazi) and peri-urban Lusaka (Chibombo, Chongwe, Rufunsa, Chilanga, and Kafue). Furthermore, PROFIT+ targeted six value chains:

maize, groundnut, soybean, sunflower, onion, and tomato, with honey added in year four. Throughout the project, PROFIT+ collaborated with a number of other USAID funded projects, including the MAWA project, Southern Africa Trade Hub and the Commercial Agribusiness for Sustainable Horticulture (CASH) project.

1.1 OVERVIEW OF ZAMBIA AND ITS AGRICULTURE SECTOR

Zambia’s economy over the past two decades has grown at an average rate of 5 percent annually.2 Despite overall positive economic growth, the agricultural sector has not kept pace with other sectors of the economy, despite its favorable endowments in terms of water resources, arable land, and climatic conditions. Agriculture’s share in the Zambian economy has declined from a range of 15-20 percent over the past fifteen years, down to a level of only 5 percent in 2018.3 Since agriculture accounts for over 60 percent of employment nation-wide and plays an even greater role in providing employment and food security for the rural population, it is generally recognized that the sector will require additional support from both public and private sources, in order to regain its role as an engine of growth, particularly in rural Zambia.

In its Sixth National Development Plan (6NDP)4, the Government of the Republic of Zambia (GRZ) confirmed its intention to promote crop diversification; increase area under irrigation; enhance productivity; and broaden utilization of improved seeds and other technologies, including mechanization.

This intention was re-stated in the Seventh National Development Plan (7NDP)5, where broad goals of increased farmer incomes and production were accompanied by specific objectives concerning the expansion of agro-industries, diversification beyond traditional field crops, and a focus on improving production of high value exports.

Consistent with these development objectives, USAID/Zambia established PROFIT+ in four districts in the Eastern Province and six districts in peri-urban Lusaka. To situate the PROFIT+ project, this section will provide an overview of production and trade trends for the selected PROFIT+ value chains.

1.2 OVERVIEW OF PRODUCTION AND TRADE TRENDS FOR PROFIT+

COMMODITIES

This section will provide a brief overview of production and trade trends for the project’s focus commodities before and during the PROFIT+ years. Table 1 shows production trends in the four staple commodities targeted by PROFIT+. Crop forecast survey (CFS) data was not available for the three horticultural crops.6 On a national level, there were increases in overall production tonnage across all four of the staple commodity groups, with significant increases registered for both soybeans and sunflowers (see Annex 4). However, for the district levels within the Eastern Province, growth was more uneven. Chipata exceeded the national rate of increase in soybean tonnage and matched the national trend for groundnut and maize tonnage, while coming in well below the average on sunflower tonnage. Katete exceeded national average volume growth on all four commodities, while Lundazi matched national growth for sunflowers and exceeded the national average on the other three crops. Petauke outperformed on soybeans but came in below the national average on maize and sunflower, and showed declining

2 USAID. Evaluation Statement of Work for Ex-Post Performance Evaluation of Production, Finance, and Improved Technology Plus (PROFIT+) Program (2012-2017), p.1 3 Ibid, p.1.

4 Government of the Republic of Zambia, National Planning Department, Ministry of Finance, Revised Sixth National Development Plan, 2013-2016, (2014), p.8 5 Government of the Republic of Zambia, Ministry of National Development Planning, Seventh National Development Plan, 2017- 2021, (2017), p.65 6 Central Statistical Office, Agriculture and Environment Statistics Division, https://www.zamstats.gov.zm/index.php/divisions/agriculture-environment-statistics production of groundnuts. Based on the limited CFS data available for peri-urban Lusaka, Chongwe and Kafue outperformed national averages on maize and groundnut. Chongwe’s production of soybean and sunflower declined over the timeframe, while Kafue matched national average for soybean but declined on sunflowers. Production for each crop per year can be found in Annex 4.

Table 1: Staples Production in PROFIT+ Focused Districts, 2009-13 vs. 2014-18

Maize (% change)

Groundnut (% change)

Soybean (% change)

Sunflower (% change)

National Zambia +13% +14% +68% +67%

Eastern Province

Chipata +12% +15% +729% +17%

Katete +20% +48% +1457% +96%

Lundazi +71% +31% +275% +65%

Petauke +2% -6% +1275% +15%

Lusaka Chongwe +21% +87% -5% -80%

Kafue +32% +151% +69% -28%

Source: Crop Forecast Surveys, 2009-2018

Figures 1 and 2 below present a summary of average import and export performance for the seven focus commodities, as captured by the UN’s Commodity Trade Statistics Database (COMTRADE), for the period from 2009-2018. All of the data represented here are derived from official sources. For maize, there was a significant 84 percent reduction in average import volumes, coupled with a 25 percent increase in average export volumes. This same pattern is repeated in the case of groundnuts and soybeans, with noticeable decreases in average imports, and significant increases in average exports. While sunflowers show a large increase in imports, the rate of export growth far exceeds that of the corresponding imports.

Export volumes of onions declined over these years, while imports quadrupled. Tomato exports increased significantly, but imports were ten times greater in volume. Of these latter value-added horticultural products, only honey showed strong growth in exports, coupled with a decline in imports.

Figure 1: Focus Crop Trade Performance, 2009 – 2018 (Imports)

Source: United Nations COMTRADE

Figure 2: Focus Crop Trade Performance, 2009 – 2018 (Exports)

Source: United Nations COMTRADE

5,000,000

10,000,000

15,000,000

20,000,000

25,000,000

30,000,000

35,000,000

40,000,000

45,000,000

50,000,000

2009 2010 2011 2012 2013 2014 2015 2016 2017 2018

Vo lu m e in k g

Key Commodities, Trade (Imports) 2009 - 2018

MAIZE GROUNDNUT SOYBEAN SUNFLOWER

ONION TOMATO HONEY COMPOSITE

1,000

10,000

100,000

1,000,000

10,000,000

100,000,000

1,000,000,000

2009 2010 2011 2012 2013 2014 2015 2016 2017 2018

Vo lu m e in k g

Key Commodities, Trade (Exports) 2009 - 2018

MAIZE GROUNDNUT SOYBEAN SUNFLOWER ONION TOMATO HONEY

Based on anecdotal evidence gathered during the course of the team’s fieldwork, there is reason to believe that considerable informal trade occurs along Zambia’s borders. While such informal trade might serve to alter these depictions considerably, it is very difficult to estimate the quantities of each crop, which are exported or imported informally in any given year.

The GRZ has over the past decade, with the support of donors and non-governmental organizations (NGOs), moved in the direction of making improvements in the agricultural economy. However, its response to many of the challenges outlined in 6NDP and 7NDP, such as unbalanced agriculture policies favoring maize production and disadvantaging the production of other crops; poor utilization of research and development (R&D), mechanization and information communication technologies (ICTs) to increase yields; and poor storage, insufficient irrigation and related infrastructure challenges, remains inadequate.

II. EVALUATION METHODOLOGY

This evaluation seeks to identify: 1) key successes and challenges of PROFIT+; 2) the merits and shortcomings, specifically related to the CAD model and the IIP Fund; and 3) why activities under these interventions have or have not continued beyond life of project. Through consultations with USAID/Zambia, this evaluation will explore the sustainability and lessons learned of key interventions under PROFIT+, based on five evaluation questions provided in the Scope of Work (SOW) (see Table 2).

7,8 Thus, this evaluation will not follow a traditional performance evaluation approach to assess whether the overall targets of the project were met but rather an assessment of the long-term effects of each project intervention since the project has closed.

Table 2: Evaluation Questions

Evaluation Questions Lessons Learned What changes have occurred in people’s lives due to PROFIT+ interventions?

What were the key drivers of successes/challenges for PROFIT+?

Sustainability To what extent did the grant facility stimulate private sector investments and expansion in the agriculture sector?

Have benefits of PROFIT+ been sustained?

How did project activities, techniques, and processes deployed by PROFIT+ address the issues of women’s empowerment in the project area?

2.1 EVALUATION METHODS

The evaluation team consisted of three core members: Team Lead, Project Manager and Agriculture Expert, as well as one local Monitoring and Evaluation (M&E) and Agriculture Specialist. LEAP III also engaged a local data collection firm, RuralNet Associates, to collect survey data on CADs to assess the sustainability and success of PROFIT+.

To address the five evaluation questions, the team utilized a mixed methods approach, combining a mix of qualitative key informant interviews (KIIs), focus group discussions (FGDs), and a quantitative survey to assess the success, challenges, and sustainability of CADs trained under the PROFIT+ project.

This approach is reflective of USAID’s similar combinations approach where two different methods are used to collect and analyze information, which is then synthesized to answer individual evaluation

7 Key interventions under PROFIT+ are as follows: CAD model, SILCs, PCs, and the IIP Fund.

8 Through consultations with USAID/Zambia, the evaluation team was informed that this should not be conducted like a traditional performance evaluation. Thus, the team proposed to structure the report by answering the evaluation questions based on the two key themes: Lessons Learned and Sustainability, revolved around CADs and the IIP Fund.

PROFIT + Evaluation Team:

Elin Cohen - Team Lead

Pin Thanesnant – Project Manager and Agriculture Expert

Mwimbu Ngoma – Local Evaluation and Agriculture Specialist

Quantitative Survey Firm:

RuralNet Associates questions.9 The evaluation team conducted desk research prior to fieldwork to identify and analyze secondary information that can be triangulated with data collected in-country.

2.2 OVERVIEW OF FIELDWORK

The evaluation team conducted fieldwork from October 21, 2019 to November 1, 2019, with the majority of the time spent in the Eastern Province. The team spent two days in Lusaka and peri-urban Lusaka (Chongwe), three days in Lundazi, five days in Chipata, and one day in Petauke. During the two-week field visit, the evaluation team carried out 39 KIIs and FGDs with the following stakeholders: PROFIT+ staff;

CADs; Saving and Internal Lending Communities (SILCs); input providers and output buyers; producer companies (PCs); and government and research institutions. Out of 39 meetings, 13 meetings were with recipients of the IIP Fund.

In addition to the evaluation team’s fieldwork, LEAP III also engaged a local data collection firm, RuralNet Associates Ltd., to carry out a survey of CADs. Based on consultations with USAID/Zambia, RuralNet carried out the surveys in the Eastern Province from October 14, 2019 to October 27, 2019.

2.2.1 QUANTITATIVE SURVEY OF CAD IN EASTERN PROVINCE

In order to understand the current state of PROFIT+’s flagship activity, the CAD model, the evaluation team developed an extensive quantitative survey instrument. The survey explored the current state of the CAD business, the type of support the CAD obtained from PROFIT+, and successes and challenges in the business. The full survey can be found in Annex 4. The evaluation team worked with RuralNet, as well as agricultural and food security, in order to administer the survey.

The evaluation team utilized the existing CAD databases supplied by USAID to establish the sampling frame for the four PROFIT+ districts in Eastern Province. The databases contained a total of 299 CADs:

79 in Chipata, 69 in Lundazi, 70 in Katete and 81 in Petauke. In the sample frame, 37 percent of CADs are women and 63 percent men. Based upon the time and resources for data collection, RuralNet administered a total of 180 quantitative surveys, which accounts for 60 percent of the total sampling frame.10 Out of the 180 interviewed CADs, 58 percent (104) participants were male and 42 percent (76) were female. Table 3 shows the number of interviews by sex and district.

Table 3: Surveyed CADs – Women and Men by District

Lundazi Chipata Katete Petauke Total

Male 60% (23) 46% (21) 70% (33) 55% (27) 58% (104)

Female 40% (15) 54% (25) 30% (14) 45% (22) 42% (76)

Total 100% (38) 100% (46) 100% (47) 100% (49) 100% (180)

2.2.2 QUALITATIVE DATA COLLECTION

FGDs: The evaluation team carried out FGDs with PROFIT+ beneficiaries in peri-urban Lusaka and the Eastern Province, specifically in Lundazi and Chipata. The evaluation team facilitated three different FGDs

9 See USAID Technical Note: Conducting Mixed-Method Evaluations, Version I, June 2013.

10 RuralNet originally pre-tested the survey questionnaire in peri-urban Lusaka, reaching five CADs to ensure that CADs understood the questions. Due to time constraints and per USAID/Zambia’s recommendation, CADs surveyed in peri-urban Lusaka during pre-testing were not included in the survey results.

for members of SILCs, which were primarily made up of women, members of PCs, and other groups that have benefited from the IIP grant. Representatives of seven PCs participated in FGDs, including ZIMBATHON, NYEBAD, SENYISA, JONAFRAPA, VISANJE, PEMMAFENG, and SIKAP.

KIIs: KIIs were carried out with PROFIT+ staff; CADs; input providers and output buyers; IIP Fund grantees; SILCs; PCs; and staff members of the Ministry of Agriculture and the Zambia Agriculture Research Institute (ZARI). Details of each group are provided below:

• CADs were interviewed to deepen our understanding of the successes and challenges they have faced in their business ventures since PROFIT+ ended. KIIs with CADs helped the evaluation team to especially target female CADs to capture their experiences. The evaluation team conducted 26 KIIs with CADs (12 women and 14 men) focused in Lundazi, Chipata, and Petauke.

• Input providers and output buyers connected with CADs were interviewed in the Eastern Province and Lusaka. The evaluation team conducted 11 KIIs in this category, meeting with Sylva Foods, JungleBeat, ZAMSEED, MRI Syngenta, Good Nature Agro, NEZI Investments, COMACO, SARO, and NANIWE.

• IIP Fund grantees were interviewed to assess the extent that the grant stimulated growth for their businesses. The evaluation team visited 13 IIP Fund grantees at their sites of work to observe the investments that were made under the grant.

• Ministry of Agriculture (MoA) staff members were interviewed, including the District Agriculture Coordinator (DACO) in Chipata and a representative of the ZARI, the largest research entity under the MoA.11

• PROFIT+ staff members were interviewed about the project to share their perspectives on successes and challenges throughout the years. The team interviewed five PROFIT+ staff including the most recent Chief of Party, as well as the most recent Deputy Chief of Party (DCOP).

2.2.3 SECONDARY DATA

The evaluation team reviewed secondary data related to agricultural production for the selected value chains and for the selected districts. The purpose of the analysis was to provide a short introduction to agricultural production in the selected districts to better understand the evaluation findings. The team also conducted a review of PROFIT+ project documents. The team worked with USAID/Zambia to retrieve project documents including the initial task order, annual performance reports, CFS data, activity work plans, CAD databases, and more. The team has summarized some of the findings in a short literature review found in Annex 3 at the end of this report.

2.3 LIMITATIONS

PROFIT+ has been closed for two and a half years. This presented a few challenges. Firstly, some people that worked for the project have left and moved away. The team even learned that two of the senior field staff have passed away. This could have affected the sampling frame to an unknown degree. USAID/Zambia staff that used to supervise the project have also moved, which made it more challenging to access information due to gaps in recordkeeping. Some project documents, particularly related to the IIP fund, 11 ZARI also received IIP grants for groundnut basic seed provision, training of seed inspectors, and basic seed multiplication.

could not be tracked down. The evaluation team obtained all the annual performance reports, but only the work plans for two years. The team has also requested project documents specific to the IIP Fund, however, no documents other than the manual were recovered. The team has therefore an incomplete picture of the IIP Fund and was unable to vet the IIP Fund history, purpose, and criteria against any project documents, and only relied on word of mouth of recipients involved.

Similarly, as more than two years have passed since the end of the project, some stakeholders did not remember how PROFIT+ supported their projects or the extent to which the support was effective. In many instances, stakeholders under PROFIT+ have also worked with many other projects funded by USAID and other donors. For instance, many CADs stated that they have worked with a variety of projects before or after PROFIT+ including CARE International, Catholic Relief Services (CRS) or Musika.

The effects of these projects will surely overlap with results from PROFIT+, making it more difficult to attribute outcomes and sustainability to this project.

Lastly, while in country, the evaluation team encountered a number of CADs that never started a business.

Prior to the evaluation team’s arrival in Zambia, the team operated under the assumption that all CADs included in the CAD databases supplied by USAID/Zambia had started an agro-dealership during the time PROFIT+ operated. As the evaluation team came to understand in the field and made evident through the results from the CAD survey, a majority of CADs included in the databases never started operating an agro-dealership. The evaluation team designed the survey questions based upon the information that was available in project documents and shared by PROFIT+ staff in KIIs. As there was no mention that many CADs had never even started a business, the evaluation team could not take this fact into consideration when designing the quantitative survey. Thus, the team had to retroactively seek explanations for why some CADs never started the business at all.

III. EVALUATION FINDINGS

Since this evaluation is focused on understanding the sustainability and lessons learned of PROFIT+, the evaluation team has structured findings under four main interventions: 1) the CAD Model, 2) SILCs, 3) PCs, and 4) IIP Fund, in order to address the five evaluation questions.12 First, the evaluation team provides an overview of how PROFIT+ was implemented in practice. This section discusses the progression of activities throughout the years, as well as shifts in management and staffing. Then, the evaluation team presents findings under the CAD model. In this section, the evaluation team explores the successes and challenges of being an active CAD, as well as CADs that never started a business, or started but failed to remain in business. In Section 3.3, and 3.4, the report focuses on the merits and issues of SILCs and PCs.

Lastly, in Section 3.5, the team explores the IIP Fund, understanding its function and to what extent it stimulated private sector investments and expansion in the agriculture sector.

3.1 IMPLEMENTATION OF PROFIT+

Based on project documents and fieldwork, the evaluation team confirmed that the implementation of the PROFIT+ project changed considerably throughout the years. At its outset, PROFIT+’s aim was to target 200,000 smallholder farmers in four districts of Eastern Province (Chipata, Katete, Lundazi, and Petauke) and three Peri-urban Lusaka districts (Chibombo, Kafue and Chongwe). In the Eastern Province, PROFIT+ was focused on all six value chains (i.e. groundnuts, sunflower, soybeans, maize, tomato and onion), while in Peri-urban Lusaka, the focus was primarily on tomato and onion.

When PROFIT+ started implementation, its main activity focused solely on the Demonstration-Host Farmer (DHF) model where lead farmers were selected to train other farmers and promote awareness about improved technologies, practices and products. One DHF worked with five lead farmers, each of which would aim to reach out to 20 smallholder farmers, with the overall goal that one DHF would ultimately reach over 105 smallholder farmers. These DHFs were then linked with seed companies such as Pannar, Seed Co., Pioneer, MRI Syngenta, and Dekalb. In the first year, 70 DHFs were selected and trained in management and agribusiness skills whereby 280 demonstration plots were established, with four types of crops per site including groundnuts, soybeans, maize, and sunflower. The DHF model was rapidly scaled up; by September 2014, PROFIT+ had established 2,780 demonstration plots.13

In March 2013, PROFIT+ also introduced the IIP Fund with the intention of dedicating $7.1 million over the life of the project.14 The IIP Fund aimed to act as a catalyst to private sector investment, to reduce risk and cost for both investors and farmers. After the first year, the IIP Fund provided smallholder farmers with various grants focused on in-kind contributions like providing sprayers, water pumps, gumboots, and other basic equipment. PROFIT+ also published online advertisements and utilized public media to get private sector entities to submit concept notes for innovative concepts or activities to be funded under

12 The evaluation team noted that several of the evaluation questions overlapped and thus, if each question were dissected and answered separately, many of the findings would be repetitive. The evaluation team suggested to USAID/Zambia to outline the report in this matter, to which it was approved for the report.

13 USAID. Production, Finance, and Improved Technology Plus (PROFIT+) Annual Performance Report No. 2 October 1, 2013 – September 30, 2014, 2014.

14 USAID. Year 1 PROFIT+ Work Plan, July 25, 2012.

the IIP Fund. By Year 3, the total funding of the project was reduced from $24 million to $18 million.15 Due to this funding change in Year 3, the IIP Fund was also “not utilized to its full capacity”.16 During the first couple of years of PROFIT+, the IIP Fund was, to a large extent, used as an operational budget for financing field activities, rather than as a fund to attract investment in agricultural businesses.

Following management issues during the first half of the project, PROFIT+ revamped its management structure in Year 3. It became clear to PROFIT+’s new management that they needed to concentrate their efforts on establishing an infrastructure to link farmers with the input and output markets, rather than engaging in agricultural extension type work. To this end, PROFIT+ shifted away from the DHF activities to focus on the CAD model whereby top performers and motivated DHFs were chosen to become CADs.

The CAD model was expected to facilitate local access to, and increase availability of, improved seed varieties and other inputs and services (such as spraying, mechanization, etc.) for the community members by building partnerships between CADs and input companies, as well as commodity buyers. CADs served as input suppliers to local farmers, but continued to demonstrate improved technologies on the demonstration plots, such as labor-saving conservation farming methods that maintain soil fertility, promotion of soil testing, methods of proper spraying, fertilizer application, etc. In Years 1 and 2, PROFIT+ developed an inventory of 123 existing agro-dealers in the project zone of influence (ZOI) and assessed their operating environment constraints. PROFIT+ additionally developed a web-based agro-dealer inventory and assessment tool to facilitate stronger linkages with input suppliers through increased transparency of agro-dealer capacity, inventory availability, record-keeping, and product lines. PROFIT+ used this assessment tool to identify 24 of the 123 agro-dealers, which had the greatest capacity and interest to partner with PROFIT+ in Year 3 on innovative business models. It should be noted that many CADs were already agro-dealers prior to PROFIT+ and distinction was not made by ACDI/VOCA in the database.

In addition to the shift in project interventions throughout the years, internal management within PROFIT+ also changed significantly. When PROFIT+ was awarded, the project was led by its first Chief of Party, however, within the first year the Chief of Party position was transferred to a local Zambian. Under his management, key stakeholders mentioned a number of issues surrounding communications and use of funds. CADs shared experiences of being told to provide funds to PROFIT+ staff in order to secure seeds but never saw the staff again after doing so. In other instances, PROFIT+ staff had promised a processor a deal in linkages to farmers, which the processor spent significant resources to prepare for, but PROFIT+ never followed through. Due to the considerable issues in internal management, management changed once again in Year 3 of PROFIT+, whereby a new Chief of Party took over the project in December of 2015. The project also hired a new finance manager, office managers, and administrative assistants.

Furthermore, the Horticulture Lead, Finance Lead, and Private Sector Lead were all no longer with the project. By Year 4, the M&E Officer, assumed the role of DCOP, as the previous DCOP left the project to become a member of the Zambian Parliament in August 2016.17 The new COP took over the project

15 USAID. Year 3 PROFIT+ Work Plan, October 2013. The project documents do not provide any explanation for the reduction in funding, but it is possible that the internal management issues contributed to the reduced budget.

16 USAID. Production, Finance, and Improved Technology Plus (PROFIT+) Annual Performance Report No. 3 October 1, 2014 – September 30, 2015, 2015.

17 USAID. Production, Finance, and Improved Technology Plus (PROFIT+) Annual Performance Report No. 4 October 1, 2015 – September 30, 2016, 2016.

at a time when significant resources had already been spent and project activities had been allowed to continue without yielding the desired result. Under his leadership, PROFIT+ made significant improvements and refocused its activities on private sector-led growth. The new COP connected promising agro-businesses with other businesses, networks, and resources and reoriented the expansive training of DHF to the more focused, and private sector-led CAD approach. Had the new COP been given the opportunity to shape and design the interventions from the beginning of PROFIT+, the outcome would most likely have been more robust and substantive.

Issues of internal management under PROFIT+ were consistently mentioned to the evaluation team as a constraint. Staffing changes throughout the years were said to be an issue, as many CADs and stakeholders were not informed. As a result, CADs did not know who to speak with, communication was inconsistent, and transitions from old staff to new staff were incomplete. Additionally, trust and past institutional knowledge were lost when CADs and other stakeholders were being introduced to new staff. Similarly, towards the end of the project, many CADs and other key stakeholders mentioned a lack of advance notice regarding the phase out of PROFIT+. By March 2017, many PROFIT+ staff members had left the project and only three remained by April 2017. While some CADs were told to attend the close out and learning conference in April 2017, many were unaware of PROFIT+’s closing. Many stakeholders mentioned not knowing that PROFIT+ was coming to an end and/or having been told to apply for a grant to simply have funds rejected and/or a lack of response all together.

3.2 THE CAD MODEL

3.2.1 OVERVIEW

During the second half of the project, PROFIT+ identified, trained, and supported farmers to launch and operate small agro-input enterprises or CADs. According to the final Annual Performance Report, there were 338 CADs18 in the final year of the project and; “More mature CADs continue to expand their operations and relationships with the Government, input companies, buyers, and financial institutions, giving the newer CADs a practical example to follow in terms of growing their rural enterprises.”19 This section explores the current state of CADs and the sustainability of the CAD businesses.

In addition to the evaluation team, RuralNet Associates simultaneously carried out a quantitative survey of 180 CADs in Lundazi, Chipata, Katete and Petauke for this evaluation. As detailed in the Evaluation Methodology section, CADs were randomly sampled from PROFIT+’s CAD databases. The survey found that only 42 percent

(76) of project participants that had been trained and supported to become a CAD under PROFIT+ was currently operating a CAD. The remaining 58 percent (104) had either never started a CAD business or started a CAD but had since closed it. 42 percent (32) of active CADs (those still operating a CAD

18 As discussed in the methodology section of this report, the PROFIT+ CAD database included 343 CADs, hence there is a small discrepancy between the stated numbers of CADs in the CAD database and the final annual performance report. The evaluation team doesn’t know the reason for this discrepancy.

19 USAID. Production, Finance, and Improved Technology Plus (PROFIT+) Annual Performance Report No. 5 October 1, 2016 – May

9, 2017, 2017, page 11.

Out of 180 CADs, only 42 percent of those trained and supported to become a CAD under PROFIT+ are currently operating a CAD business; 34 percent never started a CAD business and the remaining 24 percent started but ceased operating their CAD business.

business) are women, and 58 percent (44) are men.20 In this section, the evaluation team will assess PROFIT+’s process of selecting CADs, and explore why such a high percentage of trained CADs are inactive (not currently operating a CAD business). Active CADs will then be discussed in more detail with further findings from the CAD survey explored below. CADs were also trained to be part of SILCs and PCs, which will be discussed further in this report.

Table 4: Active and Inactive CADs To-Date (as of October 2019)

3.2.2 CAD SELECTION PROCESS

There were no clear selection criteria for CADs, leading to inconsistent results. During the first half of PROFIT+, the project focused on training and supporting farmers to establish demonstration plots. By September 2014, PROFIT+ had established 2,780 demonstration plots.21 By 2015, PROFIT+ not only changed leadership, but shifted its project focus away from the DHF model and introduced the CAD model. As the CAD approach was introduced mid-way through the performance period, there was a sense of urgency to launch the CAD activities. As PROFIT+ had already invested significant resources in the training of several thousands of demonstration host farmers, PROFIT+ focused on demo-host farmers for the initial recruitment of CADs. By September 2015, PROFIT+ reported that the project had “graduated a group of 200 best-performing farmers from their initial roles as demo-host farmers (DHF) to CADs”.22

Almost all of the CADs that are currently operating an agro-dealership (96 percent) had a demo plot during PROFIT+.23 CADs that joined PROFIT+ during the first two years of the project (2012-2014) received sustained training for a longer period of time, and as a result are more likely to still have an active CAD (see Table B in Annex 5).

While the majority of CADs had been DHFs, there were also farmers that had not, but joined PROFIT+ towards the end of the project period. For instance, the evaluation team met with two CADs that joined PROFIT+ in 2016 after their church group and women’s village group respectively recommended them for CAD training; one of them was entrepreneurial and operated a flourishing business, while the other one had very few prospects of ever starting a CAD business. PROFIT+ also integrated already existing

20 See table A in Annex 5. Note that these percentages also correspond to the percentage of women (42 percent) and men (58 percent) in the sample.

21 USAID. PROFIT+, Annual Performance Report No 2, 2014.

22 USAID. PROFIT+, Annual Performance Report No, 3, 2015.

23 Almost all (96 percent) of the CADs currently operating an agro-dealership reported that they still use the demonstration plot.

74 percent said they used it once per year; 20 percent two to six times per year; and the remaining 5 percent twelve or more times per year.

District Active CADs Inactive CADs Total

Lundazi 53% (20) 47% (18) 100% (38)

Chipata 41% (19) 59% (27) 100% (46)

Katete 26% (12) 74% (35) 100% (47)

Petauke 51% (25) 49% (24) 100% (49)

Total 42% (76) 58% (104) 100% (180)

CAD in the area from other USAID projects into the project. 37 percent of CADs surveyed for this assessment currently operating an agro-dealership had started the business without the support of

PROFIT+.

The combination of a sense of urgency to launch the CAD activity, and the fact that PROFIT+ already had poured significant resources into training DHFs, caused PROFIT+ to select “the best” DHFs to become CADs. However, there were no set criteria or guidelines for what the “good or the best” constituted. Even basic requirements, such as owning a storefront, in order to become a CAD were overlooked. CADs could not obtain the necessary licenses and permits to sell inputs unless they had a shop to sell their products. Unlike some other types of businesses, agricultural inputs should not be sold from home due to hazardous materials, such as chemical treatment on seeds, fertilizers, or pesticides. As a result, to open an agro-dealership, CADs needed capital to build or rent a shop. Thus, access to a store or the ability to fund the construction or rent of a shop should have been one of the selection requirements. In addition, entrepreneurial drive and the ability and integrity to repay input suppliers are other qualities that PROFIT+ should have considered before picking CADs. During this evaluation, the team talked to several people that were selected and trained as CADs but did not in fact start any business or move forward whatsoever.

3.2.3 INACTIVE CADS

The majority of participants trained to become a CAD under PROFIT+ (58 percent) are not operating an agro-dealership. The majority of inactive CADs never started a business (59 percent), while the remaining failed in their business venture (41 percent). 42 percent of the inactive CADs are women, and 58 percent are men, which corresponds to the overall percentage of women and men in the sample (Table A in Annex 5). 61 CADs, that is 59 percent of inactive CADs and 34 percent of all surveyed CADs, never started a CAD business. As shown in Table 5 below, the vast majority of those that never started a CAD business (89 percent) do not have a shop and therefore would have had to rent or build one prior to starting their business, which represents a significant additional cost.

In very few instances, PROFIT+ provided CADs with seed funding to construct a shop, but in most instances, CADs had to self-finance the establishment of a shop. Lack of access to capital is the main reason why inactive CADs have not been able to build or rent a shop. Some of them were never in a position to generate resources to be able to build or even rent a shop. Many inactive CADs noted that had they received some seed capital to build the shop, they believe they would have been able to operate a CAD. A few inactive CADs that never started their business (11 percent) built a shop, but due to the financial undertaking to build the shop and register (or partly register) the business, they had no capital left to start operating the business. One inactive CAD from Chipata shared “I received training and obtained the papers from PACRA and Zambia Revenue Authority (ZRA)24, but I did not get a grant or seed money from PROFIT+ to start the business. The building meant for the shop is now a dwelling house”.

24 The Patents and Companies Registration Agency (PACRA) issues business licenses and Zambia Revenue Authority (ZRA) issues tax IDs.

Table 5: Inactive CADs and Ownership of a Shop

Started CAD business but closed it down

Never started CAD business

Total

Own a shop 75% (32) 11% (7) 37% (39)

Don’t own a shop 25% (11) 89% (54) 63% (65)

Total 100% (43) 100% (61) 100% (104)

Forty-three CADs, that is 42 percent of inactive CADs and 24 percent of all surveyed CADs, started an agro-dealership but closed it down. Lack of capital to operate the business was the most commonly cited reason for why they stopped operating the business. Some noted that the business was not profitable or that there was a lot of competition in the area. Many operated on small margins and they had no financial buffer for unforeseen events, such as health issues or death in the family. To address such emergencies, they diverted money away from the business to the family and, as a consequence, had to close down the business. One female CAD in Chipata who closed her shop shared “I opened a shop, PROFIT+ trained me well and the business was running well until I got sick and spent much time at home away from the shop, we also had a funeral and that took money. Right now, I have another bereavement.” Another CAD shared that she had to sell the roof of her shop to pay for her baby’s funeral.