Attachment J.29 PMA PPERFORMANCE EVALUATION PRESENTATION.pdf

10/28/2020 PERFORMANCE EVALUATION 1

PRODUCERS TO MARKET

ALLIANCE ACTIVITY EVALUATION

Performance Evaluation

10/28/2020 PERFORMANCE EVALUATION 2

GENERAL INFORMATION

Producers to Markets Alliance

MEL EVALUATION

10/28/2020 3

PMA

PERFORMANCE EVALUATION

Contract Information Activity Name: Producers to Markets Alliance (PMA).

Performance Period: 03/03/2017 - 03/02/2022 Contractor FINTRAC Inc.

Total Cost: $71,463,707 Contract Number AID-514-C-17-00002 Mechanism: Cost-Plus-Fixed-Fee (CPFF) Contract

10/28/2020 4

COVERAGE

PERFORMANCE EVALUATION

5 regionals - 8 departments - 87 municipalities:

•Meta (9)

• Bajo Cauca: Córdoba (5), Antioquia (16) and Bolívar (6) •Caquetá (16) •North of Cauca (17) and South of Valle del Cauca (2) •Nariño (16)

10/28/2020 5

PMA’S OBJECTIVES

PERFORMANCE EVALUATION

PURPOSE:

Improved competitiveness of rural producers to respond to new and expanding market opportunities.

SUB PURPOSE 1:

Improved Domestic and

Export Market Integration

SUB PURPOSE 2:

Expanded Value Chain

Production

SUB PURPOSE 3:

Improved Market Efficiency through Infrastructure

SUB PURPOSE 4:

Learning Knowledge Management and Communications

1.1: Increased profile of Colombian products on international markets

2.1: Increased access to and adoption of technologies and good agricultural practices

3.1: Improved tertiary roads 4.1: Shared practical information and lessons learned

1.2: Increased compliance with market standards

2.2: Expanded access to quality extension and other business development services

3.2: Improved local capacity to establish and maintain roads and productive infrastructure

1.3: Expanded market linkages between producers & markets

2.3: Increased capacity of producers' organizations

3.3: Expanded investment in production, post-harvest, and market infrastructure

CROSS-CUTTING THEMES:

Social inclusion (Women, Afro-Colombian, Indigenous, Youth, LGBTI, Disabled); Partnerships;

Collaboration with Complementary USAID Investments; Capacity Building; Environment; Resiliency; Sustainability

10/28/2020 6PERFORMANCE EVALUATION

PMA SUPPORTED VALUE CHAINS

Cacao Coffee Rubber

Dairy Produce Shrimp

10/28/2020 7

EVALUATION’S OBJECTIVES

PERFORMANCE EVALUATION

Assess various aspects of the PMA Activity

Provide useful insights to the current and future programing

Evaluate the performance and effectiveness of the implemented model and the extent to which the development intervention affects and is affected by the regional context

Understand the grade in which current progress is outlining the basis for future sustainability

10/28/2020 8

EVALUATION QUESTIONS

PERFORMANCE EVALUATION

How effective are the models used by the Activity to provide technical assistance at the value chains supported by USAID?

In case the technical models used by the Activity to strengthen the value chains supported by USAID are found effective, what are the causes for those results?

To what extent is the Activity helping producers respond to market opportunities and improve the competitiveness of rural producers?

How the performance of PMA reflected in sales, productivity, and exports of farmers supported are explained in comparison to the local, regional and national context?

How is the PMA intervention affected by the local and regional illicit economies at targeted municipalities? How does the PMA intervention affect those economies?

6 What social and economic effects can be identified from tertiary roads improvement different than connectivity to market hubs?

10/28/2020 PERFORMANCE EVALUATION 9

EVALUATION METHODOLOGY

Producers to Markets Alliance

10/28/2020 10PERFORMANCE EVALUATION

FIELDWORK DESIGN

Evaluation Team

Municipalities Caquetá: Belén de los Andaquíes, Cartagena del Chairá, La Montañita, Puerto Rico y San Vicente del Caguán.

Cauca: Corinto/Miranda, El Tambo y Santander de Quilichao.

Valle del Cauca: Florida y Pradera.

Córdoba: Tierralta y Valencia.

Antioquia: Valdivia.

Nariño: Tumaco.

Bolívar: Simití y Santa Rosa del Sur.

Meta: Mesetas y Vistahermosa.

Stakeholders

• Representative –

Demonstrative Plot

• Representative –

Commercial Ally

• Representative –

Local Government

• Technical Assistant

• Representative –

Local Community

• Representative - Association

• Member National Technical Team

• Member Regional Technical Team

• Member Local Technical Team

4 teams of interviewers 3 specialists per team

(1 agribusiness specialist y 2 social specialists)

4 routes 18 municipalities

10/28/2020 11PERFORMANCE EVALUATION

FIELDWORK DESIGN

• Route 1:

Caquetá

• Route 2:

Cauca and Valle del Cauca

• Route 3:

Córdoba, Antioquia and Nariño

• Route 4:

Bolívar and Meta

10/28/2020 12

FIELDWORK DESIGN

Quantitative Interviews: 55

Qualitative Interviews: 204

Route 1: Caquetá Route 2: Cauca y Valle del Cauca Route 3: Córdoba, Antioquia y Nariño Route 4: Bolívar y Meta

10/28/2020 PERFORMANCE EVALUATION 13

QUESTION 1

Findings and Conclusions

10/28/2020 14

Q1 – EFFECTIVENESS OF MODEL

FINDINGS AND CONCLUSIONS

PERFORMANCE EVALUATION

• In general, the Activity’s models are effective in assisting USAID-supported value chains.

• The increase in production and sales is a useful indicator to estimate the technical assistance model’s effectiveness.

• The main benefits perceived by male and female producers include an augmented income and an improved quality of life.

Satisfaction with PMA Activity Gender Disaggregated

FINDINGS AND CONCLUSIONS

10/28/2020 15PERFORMANCE EVALUATION

• There is evidence of an increase in the beneficiaries’ profits from agricultural production and their regions’ agricultural diversification.

• The producers recognize improvement due to the technical assistance received from the activity and show satisfaction with its usefulness.

• Technical assistance helps to create new business alliances and to reactivate, consolidate, and strengthen existing ones.

Receptivity to Technical Assistance

10/28/2020 16PERFORMANCE EVALUATION

• The producers show satisfaction with the current trade agreements.

• Access and use of productive assets augment the benefits of technical assistance, producing more significant results.

• Availability and use of productive assets promote “associativity” and access to markets.

ASSOCIATIONS AND TYPE OF ASSET

ASSETS USE

Tool benches and machinery 58%

Post-harvesting transformation stations

83%

Storage centers 54%

10/28/2020 PERFORMANCE EVALUATION 17

QUESTION 2

10/28/2020 18

Q2 – CAUSES OF EFFECTIVENESS

FINDINGS AND CONCLUSIONS

PERFORMANCE EVALUATION

• Filters and entry criteria allow the activity to make an adequate selection of beneficiaries, choosing participants with sufficient characteristics to develop a productive project and apply the technical assistance service.

• PMA’s use of different technical assistance delivery mechanisms has allowed an adequate provision of the service. Among these mechanisms, training on productive assets and training on good practices stand out.

Successful Elements

MATURITY OF ASSOCIATIONS

CHARACTERISTIC % OF TOTAL

Associations with 10 or more years 65%

Associations with bylaws 96%

Associations with accounting records 69%

Associations with business plans 58%

Associations with own headquarters 45%

Improvable Elements

10/28/2020 19PERFORMANCE EVALUATION

• While there is no model to service technical assistance to producers; there are a battery of mechanisms or tools to transfer knowledge.

• There is no operating manual or document that explains (i) the sequence to use the assistance mechanisms or (ii) the criteria for choosing them.

• There are a limited number of the activity's technical assistants to provide technical services (a few visits or many producers per trainer).

Q2 – CAUSES OF EFFECTIVENESS

FINDINGS AND CONCLUSIONS

“In the case of Simití, there is only one technician for every 300 cocoa producers; if you visit one daily, you will not be able to visit all of them in one year. In the case of Valdivia, in most projects, 20 to 30 visits per producer are carried out in a truly short period.”

Semi-structured interview, Bolívar

10/28/2020 20PERFORMANCE EVALUATION

• The selection of producers who are also technical assistants allows efficient knowledge sharing.

• The frequency of assistance varies according to the value chain.

• Grouping associations together increases the impact of technical visits.

• When the commercial partner pays extra for the product's added value, the producer is encouraged to continue applying the technical assistance models.

Q2 – CAUSES OF EFFECTIVENESS

FINDINGS AND CONCLUSIONS

Other Elements

ASSOCIATIONS WITH SALES AT PREMIUM

PRICES, 2020

VALUE CHAIN ASSOCIATIONS

Cacao 93%

Coffee 100%

Rubber 100%

Dairy 100%

10/28/2020 PERFORMANCE EVALUATION 21

QUESTION 3

10/28/2020 22

Q3 – COMPETITIVENESS

FINDINGS AND CONCLUSIONS

PERFORMANCE EVALUATION

• Different individuals working with production chains acknowledge the importance of participating in international cooperation projects.

• Different types of production chains recognize the importance of consolidating product volumes to access markets.

• The associations embrace the technical assistance service for post-harvest tasks.

ASSOCIATIONS MENTIONING

TRAINING TOPICS

TOPIC ASSOCIATIONS

Crop Management 84%

Good Practices 96%

Soil Management 95%

Post-harvest 77%

Pests and Diseases 91%

10/28/2020 23PERFORMANCE EVALUATION

• Producers perceive benefits in their crops because of improvements to local roads.

• Most of the association representatives state that they are satisfied with the purchase and marketing agreements.

• The associations supported by PMA have a better performance in the organizational capacity index.

Associations with Purchase Agreements

Q3 – COMPETITIVENESS

10/28/2020 PERFORMANCE EVALUATION 24

QUESTION 4

10/28/2020 25

• PMA’s beneficiaries have lower or equal yields than farmers with similar crops in the rest of the country; security conditions and illegal economic activities explain this phenomenon.

• The municipalities in which PMA operates have had a relative reduction in illicit crops compared to the rest of the country.

PERFORMANCE EVALUATION

Q4 – CONTEXT COMPARISON

FINDINGS AND CONCLUSIONS

LOGARITHM – COCA CULTIVATED HECTARES

YEAR PMA

MUNICIPALITIE

S

NON-PMA

MUNICIPALITIE

S

DIFFERENC

E

2015 5.417 4.133 -1.284**

2018 5.690 4.834 -0.856

10/28/2020 26PERFORMANCE EVALUATION

Hectares with Coca, PMA Municipalities vs. Non-PMA Municipalities

Q4 – CONTEXT COMPARISON

28/10/2020 27Cultivos legales de cacao compiten con cultivos ilícitos.

28/10/2020 28Autocorrelación Espacial Bivariada

28/10/2020 29Autocorrelación Espacial Bivariada

28/10/2020 30Autocorrelación Espacial Bivariada

28/10/2020 31

FINDINGS AND CONCLUSIONS

• PMA’s beneficiaries see value chains as a viable alternative to illicit crops.

• The Activity appears to be displacing the illegal economy

10/28/2020 32

• The reduction of illegal crops in the areas where PMA operates has increased the population's sense of security.

• Most beneficiaries have access to a road less than a kilometer away, which improves the opportunities for marketing of products and safety.

Distance to Road – PMA Beneficiaries

Q4 – CONTEXT COMPARISON

FINDINGS AND CONCLUSIONS

Areas with presence of PMA beneficiaries Distance to roads in Kilometers

<0.1

0.1 – 0.9 1 – 3 4 – 9 10 – 80

10/28/2020

• PMA crops have lower or equal yields than those presented by similar products in the rest of the country and areas serviced by PDETS.

• The activity participants acknowledge USAID’s continued presence in the region and recognize the PMA.

PMA Crops vs. Other Crops

10/28/2020 PERFORMANCE EVALUATION 34

QUESTION 5

10/28/2020 35PERFORMANCE EVALUATION

• PMA promotes hubs of licit economic activities in the regions, improving security conditions and strengthening relations among the residents.

• The productive legal options offered by PMA are the supported value chains.

• Strengthened marketing channels enhance the sustainability of licit agricultural processes.

Q5 – ILLICIT ECONOMIES

FINDINGS AND CONCLUSIONS

Semi-structured interview, Local official, Caquetá

The farmer already understands that the best thing is to be legal and produce a product where he can walk with his [or her] head held high.

There begins to be a peace of mind in the producer when he [or she] can radiate a positive example to his (or her) family and environment.”

10/28/2020 36PERFORMANCE EVALUATION

• The implementation of PMA's technical support (demonstration plots, training associations, technical visits, and field days) increases solidarity among the neighbors.

• The construction of regional roads strengthens the community's trust in local authorities and promotes a sense of belonging.

Q5 – ILLICIT ECONOMIES

FINDINGS AND CONCLUSIONS

Semi-structured interview, Demonstration plot, Cauca

It seems to me that PMA applies a comprehensive model that changes the producer's mentality in terms of strengthening the families and community organization around the agricultural activity. All this has empowered the small farmer. Learning by doing, a methodology used in demonstration plots, field days where we all meet, and technical assistance, these activities strengthen ties between communities, ties that were previously weak, or did not exist.”

10/28/2020 37PERFORMANCE EVALUATION

• Without the involvement of young males and females in agricultural activities, the continuity of the sector, transfer of knowledge, and sustainability are at risk.

• State agencies and local governments have seen PMA as a viable model of an agricultural program.

Q5 – ILLICIT ECONOMIES

FINDINGS AND CONCLUSIONS

Semi-structured interview, Local official, Valle del Cauca

I think there is a relationship (between PMA and the mayor's office):

PMA staff have been extremely interested in learning about the PDET issue, the development plans with a regional focus. I would think that they are very well versed on the subject. In the north of Cauca and Valle, PMA has been a good learning experience for the communities; we hope that the same approach can be used in the projects chosen by the PDETs.”

10/28/2020 PERFORMANCE EVALUATION 38

QUESTION 6

10/28/2020 39

• PMA's interventions with tertiary road improvements in targeted municipalities has brought positive social and economic effects.

• Farmers report decreases in travel times when moving their products thanks to the improvement of the tertiary road network.

• Producers report a reduction in travel prices and freight rates thanks to the upgrades of the tertiary road network.

Q6 – TERTIARY ROADS

FINDINGS AND CONCLUSIONS

PERFORMANCE EVALUATION

BENEFITS ASSOCIATED WITH ROAD IMPROVEMENTS,

WHAT BENEFITS DO YOU PERCEIVE, OR WILL YOU

HAVE WITH THE IMPROVEMENTS OF ROADS?

Decrease in travel times 38.1%

Decrease in travel costs 33.6%

Increase in transportation means 32.0%

Better products arriving at markets 29.5%

Valuation of real state 45.9%

Improves access to the farm 34.0%

Motivates other farm improvements 29.0%

Motivates community to keep road improvements 27.6%

Motivates local governments to improve road network 27.4%

10/28/2020 40PERFORMANCE EVALUATION

• Producers report an improvement in the quality of their products and delivery times due to the use of better vehicles.

• Enhancements to the tertiary road network have allowed the use of the roads during any season and provided access to new transportation companies.

• Producers have observed changes in the type of vehicles that transport products, from smaller to larger.

Q6 – TERTIARY ROADS

FINDINGS AND CONCLUSIONS

Semi-structured interview, Meta

If you could get out of there, you would be stuck, locked in, and couldn’t get out. Now, our families and relatives have small cars and come from the families and can get to our farms and visit us and everything is good.”

10/28/2020 41PERFORMANCE EVALUATION

• After road improvements, farmers can use vehicles to their maximum capacity.

• Road improvements have brought new economic activities such as tourism.

• The communities were committed to maintaining the roads using different mechanisms.

Q6 – TERTIARY ROADS

Type of Transport Used by Department

10/28/2020 PERFORMANCE EVALUATION 42

FINAL REPORT

RECOMMENDATIONS

10/28/2020 43

TECHNICAL WORK

PERFORMANCE EVALUATION

•Develop an operational manual for the PMA activity, with specific sections for each component and its services.

•Write down the technical assistance model for producers; this document can be sorted by typologies of associations, municipalities, or production chains.

10/28/2020 44

OPERATIONS

PERFORMANCE EVALUATION

• Define a route to certify associations as EPSEAS and improve technical assistance in GAP, GHP, GFP, and GMP.

PMA can also promote written commercial agreements.

• Strengthen the work done with commercial allies to guarantee the continuity of improvement once the PMA activity finishes.

• Strengthen the infrastructure component to include logistics; this component would have a significant impact on competitiveness and market opportunities. Consider reinforcing the concepts of tracing and tracking.

10/28/2020 45

OPERATIONS

PERFORMANCE EVALUATION

• Increase awareness and train male and female farmers and associations on the importance of transportation costs to increase the economic return from road improvements in the regions.

• Guarantee that all the tasks implemented by the infrastructure component strategically link to initiatives undertaken by the other two components.

• Create technical committees by region and value chain to follow-up on the agreements with commercial allies and associations.

10/28/2020 46

M&E

PERFORMANCE EVALUATION

Review the methodological design and the sample of data compiled by the PMA activity to respond to its information needs.

For the farmer’s characterization of 2021, review and enlarge the instrument’s scope.

Consolidate and update a stakeholders' database.

Include indicators by component and create templates to record the follow-up results.

10/28/2020 47

SUSTAINABILITY

PERFORMANCE EVALUATION

• Encourage male and female youth as much as possible to participate in the

PMA.

• Strengthen the relations between the PMA and the Post-Conflict Office, ART, and local governments to implement PDETs and the National Comprehensive Program for Crop Substitution.

• Exchange regional experiences and knowledge among value chains and types of associations.

10/28/2020 48

Ana María Rivera Chief of Party

Carrera 7 No. 74B-36, Piso 6, Bogotá, Colombia email: arivera@panagoracolombia.net

Orlando Gracia Lead Evaluation Expert

Carrera 7 No. 74B-36, Piso 6, Bogotá, Colombia email: ogracia@panagoracolombia.net

PERFORMANCE EVALUATION

mailto:arivera@panagoracolombia.net mailto:ogracia@panagoracolombia.net