Attachment J.10 Impact and Performance Evaluation RFI.pdf

DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government.

IMPACT AND PERFORMANCE

EVALUATION OF THE USAID/COLOMBIA

RURAL FINANCE INITIATIVE (RFI)

FINAL EVALUATION REPORT

December 2021

MEL ACTIVITY

Contract Title: USAID/Colombia Monitoring, Evaluation and Learning Activity

Contract Number: 72051419C00001

Submitted: December 18, 2021

Contractor: Panagora Group, Inc.

COR: Omar López

Authors: Allison Benson, Camila Barrera, Paula Zuleta, Santiago Tobón, Simón Borrero, Carol Pinzón, Natalia Rodríguez, Orlando Gracia, Luisa Fernanda Cardona, Juan Guillermo Bedoya, and Irene Velez

Panagora Group Contacts:

Ana María Rivera Chief of Party Carrera 7 No. 74B-36, Piso 6, Bogotá, Colombia Email: arivera@panagoracolombia.net

Orlando Gracia Lead Evaluation Expert Carrera 7 No. 74B-36, Piso 6, Bogotá, Colombia Email: ogracia@panagoracolombia.net

Francis Okello, PhD Director, Monitoring, Evaluation, Research, Learning and Adapting Practice 8601 Georgia Avenue, 8th Floor, Silver Spring, MD, USA Email: francisokello@panagoragroup.net mailto:arivera@panagoracolombia.net mailto:ogracia@panagoracolombia.net mailto:francisokello@panagoragroup.net i | FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE USAID.GOV

CONTENTS

ABSTRACT III

EXECUTIVE SUMMARY 1

EVALUATION FINDINGS 2

EVALUATION RECOMMENDATIONS 8

EVALUATION PURPOSE 12

ACTIVITY DESCRIPTION AND BACKGROUND 13

EVALUATION QUESTIONS AND METHODOLOGY 15

EVALUATION QUESTIONS 15

METHODOLOGY 16

EVALUATION FINDINGS AND CONCLUSIONS 20

QUESTION 1 20

QUESTION 2 26

QUESTION 3 40

QUESTION 4 42

QUESTION 5 44

QUESTION 6 45

QUESTION 7 49

EVALUATION RECOMMENDATIONS 54

ACCESS TO APPROPRIATE FINANCE SERVICES 58

COSTS OF FINANCIAL SERVICES 60

ANNEX 1: RECOMMENDATIONS IMPLEMENTATION MATIRX 66

ANNEX 2: SUPPLEMENTAL ANALYSIS AND DATA 84

ANNEX 3: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 298

ANNEX 4: DETAILED METHODOLOGY 349

ANNEX 5: INSTRUMENTS 389

ANNEX 6: SOURCES OF INFORMATION 471

ANNEX 7: EVALUATION STATEMENT OF WORK - SOW 477

ANNEX 8: EVALUATION STATEMENT OF WORK - SOW 497

ANNEX 9: SIGNED DISCLOSURES OF CONFLICTS OF INTEREST 500

USAID.GOV FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE | ii

ACRONYMS

BC Banking Correspondent

BdO Banca de las Oportunidades

CONPES National Economic and Social Committee

COP Colombian Peso

DCA Development Credit Authority Loan Guarantees

DID differences in differences

FE Financial Education

GLACS Lending and Saving local groups

GoC Government of Colombia

FI Financial Institution

MADR Ministry of Agriculture and Rural Development

MEL Monitoring, Evaluation, and Learning

RFI Rural Finance Initiative

SFC Superintendence of Finance of Colombia

TA Technical Assistance

USAID United States Agency for International Development

VCF Value Chain Finance iii | FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE USAID.GOV

ABSTRACT

The Rural Finance Initiative (RFI) aims to expand access to financial services in Colombia´s rural regions.

The initiative, established in 2015 and covering 197 municipalities, works side-by-side with financial and non-financial institutions, providing technical and financial assistance (grants) to:

• Improve rural financial intermediation

• Reduce the barriers that rural inhabitants face for accessing rural financial services

• Modernize the financial environment

• Implement learning, knowledge management, and communications strategies to identify, document, systematize, and share best practices and lessons learned

The purpose of this evaluation is to assess whether the RFI’s interventions, deliverables, and outcomes were accomplished according to the work plan and theory of change and to what extent they are relevant based on the current national and local contexts. The evaluation comprises two components:

impact and performance evaluations, both relying on mixed methods. The impact evaluation is mainly based on a Differences in Differences quasi-experimental approach with data from a survey of 3.047 households, complemented by the analysis of secondary data from the program and government sources. The RFI performance evaluation is based on the analysis of i) 109 interviews conducted at the national and local levels with financial institutions and rural inhabitants; ii) a digital survey of 322 local bank branches; and iii) project monitoring data and relevant qualitative documentation. We identified over 250 findings, leading to 60 conclusions and 41 recommendations.

The main conclusion from this evaluation is that the combination of financial and technical support from RFI contributed to (i) accelerating the establishment of financial institutions (FIs) in rural areas through traditional and non-traditional channels, (ii) the improvement of internal processes of FIs, (iii) the adoption of digital technologies, and (iv) the adaptation of financial products (mainly credit) to the specific needs of the rural market. However, RFI generated little change in current savings and insurance mobilization trends at the aggregate (municipality) level. Opportunities remain for more directly orienting future support towards financial inclusion results, aligning incentives to privilege goals related to capturing new clients and the incremental financial services and resources offered to these.

1 | FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE USAID.GOV

EXECUTIVE SUMMARY

Increasing access to appropriate financial services remains a challenge for rural development in Colombia, especially in remote rural areas that are key for peace building. Therefore, reducing barriers to credit access, promoting savings and insurance, and making money transfers more efficient and less costly are relevant policy objectives with the potential of improving livelihoods of millions of micro, small, and medium-sized rural producers and businesses, and for fostering the economic growth of the agricultural sector.

The RFI, financed by the United States Agency for International Development (USAID) and implemented by Chemonics International, aims to expand access to financial services in Colombia´s rural regions affected by social and economic consequences of the internal conflict and deep inequalities. The initiative, established in 2015, works side-by-side with financial and non-financial institutions at the national and local level in 197 municipalities, strengthening their capacity to better service rural entrepreneurs with client-responsive financial products. To accomplish this, RFI provides technical and financial assistance (grants) to FIs. Another component of the initiative is working with regulatory and other financial sector actors to identify and implement regulatory changes to modernize the financial environment. Finally, the initiative implements learning, knowledge management, and communications strategies to identify, document, systematize and share best practices and lessons learned to relevant stakeholders, including the Government of Colombia (GoC), communities, and the private and third sectors.

The RFI activity impact and performance evaluation was conducted by the USAID / Colombia’s Monitoring, Evaluation, and Learning (MEL) Activity. The impact and performance evaluation seeks to facilitate informed program management decisions and shape the long-term strategic design and implementation of programs and decision-making, contributing to USAID’s knowledge base to improve development outcomes. Impact evaluations aim to measure changes in development outcomes resulting from the implementation of a specific intervention, measured using models of cause and effect with credible and rigorously defined counterfactuals. In turn, performance evaluations address questions such as what a particular project or program has achieved; how it is perceived and valued; and whether expected results are occurring1. In this evaluation, the impact component analyses the extent to which the RFI improved the efficiency, effectiveness, and quality of the portfolio of financial services, how it reduced the costs of financial services, how it increased vulnerable populations access to appropriate financial services, and what underlying variables mitigate or enhance the initiative´s impact. In addition to addressing these questions, the performance evaluation identifies the lessons learned from the original design towards attaining the Initiative´s goals, how the implementation strategy contributed to the activity goals, and to what extent the Covid-19 pandemic affected implementation.

This evaluation relies on a mixed methods approach to answer the above questions, synergistically integrating quantitative and qualitative analyses. The quantitative analysis relies on data from: i) an onsite survey of over 3.047 households; ii) a digital survey of 322 local bank branches; and iii) secondary data from the Initiative´s Monitor data, and administrative sources. The digital survey data was analyzed and merged with administrative sources. In turn, with the on-site survey of households, a quasi-experimental

1 USAID Evaluation Policy document “Learning from Experience”

USAID.GOV FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE | 2

analysis with a Differences in Differences (DID) approach was conducted, comparing five treatment and five control municipalities between the baseline (2017) and the follow-up (2021). This analysis was complemented with a cross section analysis using Logit estimations, a fixed effects model, and a matched DID estimation. Additionally, we conducted a geographic information systems (GIS) analysis of the location of interviewed households and banks and estimated linear distances as a source of variation.

Finally, the qualitative analysis was carried out based on data collected from 109 interviews in 10 municipalities exhibiting varying degrees of financial development and program intensity. Key informants included staff from national and local FIs, regulatory actors, local leaders, and rural inhabitants. The qualitative data was transcribed, codified, analyzed and visualized.

The different results from the RFI evaluation were integrated, providing over 250 findings leading to 60 conclusions and 41 recommendations. The main results are organized by evaluation question. They are summarized below, while further detail is provided in the report´s main body and in the Annexes.

EVALUATION FINDINGS

QUESTION 1: IMPROVEMENTS IN THE EFFICIENCY, EFFECTIVENESS, AND QUALITY OF THE PORTFOLIO

OF RURAL FINANCIAL SERVICES

The financial incentives (grants) provided by RFI contributed to Financial Institutions (FIs) mobilizing COP 3.56 billion in focalized areas, characterized by low levels of financial penetration. Resources were mobilized through different types of FIs, ranging from cooperatives, microfinance institutions and niche banks reaching small producers, to large banks (mainly reaching large/medium-sized producers).

A majority (83%) of the offices of participating FIs in the targeted areas reported having introduced changes in their internal processes, including the simplification of procedures and requirements of financial services (mainly in credit products). The offices that changed their processes achieved significantly better results in the average number of new clients and at-risk portfolio.

Through RFI´s support, the FIs adapted their loan products to the needs of the rural client, including adjusting products to production cycles, payment periods, and requirements. In addition, FIs adjusted their credit policies and risk assessment models based on support from RFI.

The Value Chain Finance (VCF) methodology design amidst RFI was innovative and had potential to improve the allocation of agricultural credit, but it was not fully implemented in most FIs. While FIs perceive it as helpful, its implementation is considered complex as it depends on the level of development of producer organizations and on the joint work between these business clients and the FIs.

Development Credit Authority Loan Guarantees (DCA) offered by USAID were a key instrument to increase credit provision. They are easy to use and benefit FIs by offering higher coverage.

FIs participating in the initiative have an average portfolio at risk index (7.3%) above the mean of the financial system (5%). However, this was not a concern for the operation of the Initiative. The results evidence higher portfolio averages at risk in municipalities with fewer offices per capita, more distance to market, less female population, fewer teachers per student, and fewer homicides per capita.

3 | FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE USAID.GOV

Adjustments and innovations in savings products were less prioritized by the Initiative than credit products. Not all RFI work plans with FIs included savings, nor were these a central component in these.

In general, rural clients still consider that they do not have real incentives to save in FIs due to the high costs (management fees and transaction costs) and low benefits (low-interest rates) relative to other traditional forms of saving such as in-kind savings.

The FIs' insurance products continue to be in general limited to personal insurance, usually linked to credit (life insurance). Few FIs offer agricultural insurance and there is no evidence of the development of parametric insurances2. While some FIs managed to offer insurance products that they did not previously offer; in general, insurance topics were not prioritized in the Initiative´s work plans.

With the support of RFI, FIs made significant progress in technological developments to make their internal processes and interaction with the client more efficient. Nearly all (95%) of the participating FI branches reported having adopted technological innovations to improve their operation. The offices that implemented technological innovations obtained better results in clients linked and in loan portfolio quality.

73% of the local FI branches report having taken actions to improve customer satisfaction. In general, high levels of satisfaction were found, with larger improvements in RFI than in control municipalities.

However, we found that only 38% of the people in RFI municipalities agree that the information provided by the FI is clear, easy to understand and complete. This percentage dropped 10 percentage points compared to the baseline, whereas in control municipalities the reduction was of 7 percentage points (from 47% to 40%=. Problems of communication relate to distrust in the financial sector.

Among clients who have accessed credit, satisfaction levels in RFI municipalities is high (86.5%), increasing by 10 percentage points, while in control municipalities it increased less than 1 pp. There is no statistically significant improvement in satisfaction with savings products.

QUESTION 2: INCREASED ACCESS OF RURAL AND VULNERABLE POPULATIONS TO APPROPRIATE

FINANCIAL SERVICES

With RFI´s support, the FIs increased the access channels for rural clients (opening of offices, banking correspondents, web pages, app development), reaching dispersed areas, not only urban centers of the municipalities, where they already operated. The percentage of municipalities with bank offices has grown in RFI municipalities and is higher (87%) than in similar municipalities not served by the initiative (76%).

The greater coverage of branches and mobile advisors resulted in a positive impact of RFI on perceptions of access and in a reduction in the proportion of people who report not having access to financial products due to access barriers such as distance to the FIs. However, the percentage of people who consider that during the last four years, it has become easier to access financial products (credit, savings accounts or insurance) is less than 50%, and there are no statistically significant differences between RFI and control municipalities.

2 Non-traditional insurance products that covers the probability of a pre-determined event happening (e.g. weather event).

USAID.GOV FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE | 4

Banking Correspondents (BCs) are relevant for expanding coverage of financial services at a lower cost.

However, their potential is limited by aspects such as the quotas and type of operations they can perform, and connectivity problems.

RFI also contributed to the expansion of virtual channels (e.g., apps, WhatsApp), which was valuable in improving customer contact, especially in the context of the COVID pandemic. However, external barriers to connectivity and digital skills persist.

During the implementation of the program, FIs reached more than 855,000 clients in the focalized municipalities with new or adjusted products or processes. There is a high variance in the number of clients mobilized by FI, with niche banks being the ones that mobilized the most (up to 63,138 per year in the case of Banco Mundo Mujer, while large banks such as Davivienda report on average 1,031 clients mobilized).

A total of 361,217 credit clients were linked to the participating FIs and COP 2.8 billion were mobilized in focalized municipalities during the implementation of the Initiative. However, the evaluation found no impact on the proportion of people who accessed credit, which remains at 27.2% in RFI municipalities, compared to 32% in control municipalities.

FI participating in RFI linked 225,388 clients to savings products. Although the number of savings accounts per capita is lower in RFI municipalities, the difference reduced by 31% during the implementation of the initiative. Nonetheless, in terms of effective use of savings products, there is a drop in RFI municipalities (from 15.4% to 10%), although this drop is not statistically different from the drop in control municipalities (from 17.9% to 13.2%). Low savings relate to the low-income levels as well as to the few real incentives for formal saving (low interest, high management fees). Furthermore, we found that distrust in the financial system increased in RFI municipalities from 9.6% to 20.5%, while in controls, the increase was only from 16.6% to 20.3%.

Although nearly 270,000 new insurance customers were reached by participating FIs (mainly with credit-related insurance) in the targeted municipalities during the RFI implementation, there was an aggregate reduction in the proportion of people who reported having insurance (from 20.8% to 5.8%). This reduction is explained by a reduction in the people who report having mandatory insurance. We also find that in general, rural clients know little about financial insurances.

Most payments continue to be made in-cash and transfers, through money transfer companies. In RFI municipalities, the percentage of money transfers made through money transfer companies fell 5 percentage points (from 94% to 89%), falling the same magnitude as in control municipalities. The participating FIs do not report specific changes or developments of new products of transfers and remittances.

Only a minority of rural inhabitants accessed financial education in the last 12 months. While in control municipalities this percentage is 1.4%, it is 2% in RFI municipalities, with a non-statistically significant reduction compared to its baseline level (3.6%). In general, less than 25% of rural inhabitants consider they have good knowledge on financial topics, although with greater and statistically significant improvements in RFI relative to control municipalities. However, financial education is still perceived as something secondary both in FI´s operation and in the RFI´s work plans. Its potential to attract and train customers is not fully developed.

5 | FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE USAID.GOV

Nearly half (48%) of the clients reached through the program were women, and we find a positive DID impact in the share of women who consider they have adequate access to FIs where they live. While less women have accessed a loan in RFI municipalities (31.3%) than in control municipalities (40.7%), their access rate is higher than men living in RFI municipalities (24.1%). This pattern also holds for access to savings accounts.

In RFI municipalities, there were advances in access to financial products for victim populations, ethnic minorities and people who have not completed secondary education. However, gaps persist, especially in terms of people with high versus low educational levels (e.g., the percentage of people who have a savings account in RFI municipalities is 17% among those without a high school education, versus 58.2% for those with one). For young vs. older clients, we also find large gaps (e.g., youth´s access to credit is 20 percentage points lower than for older clients, and the gap only decreased in control municipalities).

Additionally, we observe an increase in the use of technology to manage savings accounts and to make digital transactions in young clients.

The positive DID coefficients found on access to savings accounts, use of digital devices and reductions in the travel costs and travel time to FIs, are explained by rural areas (i.e., the DID coefficients are positive and statistically significant for rural areas, whereas they are not significant for urban ones).

QUESTION 3: REDUCTIONS IN THE COSTS OF RURAL FINANCIAL SERVICES

The offices that took measures to reduce operating costs (66% of all participating branches) obtained better results in the number of clients reached, the share of women clients, and portfolio quality. Cost reductions were achieved by digitalizing processes, simplifying procedures, digital banking, expanding coverage through BCs and employing innovative instruments such as DCA guarantees.

The increase in FI coverage translated not only into better perceptions on access, but also, on lower access costs for the clients. We find a positive and statistically significant DID coefficient on the transport cost for reaching FIs and on the average travel time to these.

However, challenges persist for translating improvements in efficiency and adjustments to products into lower product costs for clients. In general, rural clients do not perceive reductions in product costs, and costs remain a barrier for access (specially for savings), as well as a barrier for making digital transactions of mass usage.

QUESTION 4: UNDERLYING VARIABLES MITIGATING OR ENHANCING THE IMPACT OF THE INITIATIVE

RFI generated the incentives that allowed FIs to reach areas with high insecurity levels, evidencing it is possible to service them while adjusting to the local realities. However, the armed conflict continues to constitute a barrier for financial inclusion and financial deepening. Citizens face a disincentive for investing and requesting credit due to the fear of losing their production or land to armed actors and not being able to repay their debt.

There is a persistent lack of trust in the financial system, and this continues to be among the reasons why people do not demand products. The lack of trust is associated, among others, to problems in communication, as well as from some specific background experiences, for example, bankruptcy of local cooperatives. There is also a lack of trust in digital banking.

USAID.GOV FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE | 6

While there have been improvements in internet connectivity (access to internet or data plans increased from 24.8% to 36.1% in RFI municipalities), connectivity is still eight percentage points below control municipalities, and problems of connectivity are common and persistent, being a barrier for digital financial inclusion.

QUESTION 5: EFFECTS OF THE COVID-19 PANDEMIC

All of the participating FI branches report drops in the number of credits allocated, savings accounts opened, and insurances provided as a result of the Covid-19 pandemic. However, 50% report increases in digital transactions. The pandemic was an incentive to accelerate digital processes and remote attention channels (e.g., attention via WhatsApp, use of Apps) and the design and use of digital products (e.g., digital credits). The support provided by RFI to incentivize digital processes and products and provide digital infrastructure (e.g., cell phones, computers, digital devices) was valuable for allowing FIs to better face the pandemic. In aggregate, the pandemic was not an obstacle for RFI goal achievement.

QUESTION 6: IMPLEMENTATION STRATEGY AND ITS CONTRIBUTION TO THE ACTIVITY GOALS

The Technical Assistance (TA) provided by RFI was relevant for supporting FIs in the development and implementation of new products and processes, as well as in the improvement of existing ones. RFI invested almost COP 7.000 million in TA, 85% of these targeted FIs, 10% the Government and the rest cross-cutting support, such as developing financial methodologies. However, there is no correlation between the investment in TA and results obtained by FIs in terms of new clients or resources mobilized.

The incentive grants were successful as an instrument to incentivize the opening of new bank branches in disperse places where some FIs report they might not have opened them or might not have opened them as fast. The financial incentives varied significantly in amount and in the results achieved (cost-efficiency). Two-thirds of the clients were reached by three FIs, while the contribution of the other 12 FIs was relatively marginal. Similarly, 80% of the financial resources were mobilized by 5 FIs, while the rest contributed with a small share each.

While RFI carried out some exploratory work with regulatory agents, this did not derive in adjustments to financial intermediation regulation. There are still regulatory changes that could favor the scalability of the innovations in financial inclusion obtained through the support of RFI (e.g. regulation on rural BC, low value payment systems, subsidized interest rates).

The alliances with local actors (e.g., social organizations and local governments) are not mentioned as central to the FI´s operation nor to the RFI. There are opportunities to leverage local actors to build trust and learn about the particularities of a context, especially when entering a new municipality.

There is no evidence of a close articulation between RFI and public programs or initiatives, despite some initial efforts made. Opportunities remain to strengthen the impact, sustainability and scaling-up of the Initiative through alliances with the local and national Governments.

The RFI was successful in documenting, systematizing and disseminating good practices, success stories and lessons learned through the preparation of documents and holding events in which knowledge management products were elaborated.

7 | FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE USAID.GOV

The communications component was not central in the work plans with FIs. There are opportunities to continue working in this area, especially with respect to communications between FIs and rural customers (including communications with a differential approach) as a strategic part of the operation of offices, especially local ones.

RFI made visible the gender component and differential approach to vulnerable populations (victims and ethnic minorities) in the report of goals and in the objectives of the work plans. However, in general, FIs see the gender and differential approach as secondary both in their work with RFI, and in their internal operations.

QUESTION 7: LESSONS LEARNED FROM THE ORIGINAL DESIGN

By organizing the Initiative´s operation around geographic corridors that are strategic for peace building, it was possible to efficiently canalize resources and TA to territories that required these. In operative terms, no major particularities are evidenced in having structured the initiative around geographic corridors.

The emphasis on working directly and one-on-one with FIs was a success, allowing the program to respond to their specific needs This approach increased FI´s interest in participating and generating the incentives and capacity required for them to continue rural financial deepening with their own resources once RFI operations ended.

The grants were tailored to each FI, and as a result, the RFI monetary contributions and the investments made by each FI in different activities varied significantly, in response to their own priorities. Among the work plans defined with the FIs, activities with a higher return (e.g., credit) were prioritized as part of the contractual design of the program, stating that at least 80% of the mobilized funds had to be productive lending. In contrast, other service areas (e.g., insurance, savings and communications), with a lower return, but which are key for rural financial development, stood out less in the work plans.

The flexible way in which grants were structured generated some difficulties in establishing minimum parameters and in comparing goals, costs and results across different FIs. This does not allow to precisely identify instances where results were achieved in a more cost-effective manner. Also, considering the level of detail of the available grant cost information, and considering that not all FI resources were auditable by RFI, it is not possible to determine whether the resources provided by FIs (counterpart resources) were incremental resources that can be attributed to the activities carried out within the RFI work plans, or whether these were recurrent FI resources, not necessarily linked with the work plans.

The FIs have excellent perceptions regarding the quality, relevance and opportunity from the TA provided by RFI. They recognize it as an innovative component of the initiative that made it attractive to a variety of FI types and that achieved concrete results in improving processes and products. FIs value that the TA was adjusted to their specific needs and business model, derived from an initial diagnosis;

this should increase appropriation of RFI activities and knowledge transfer.

The FIs have continued serving rural populations, and in general, have continued with the implementation of processes and products started within RFI. In some cases, FIs replicated experiences of other FIs and offices. The installed capacity left by the TA, and the alignment of incentives achieved

USAID.GOV FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE | 8

through the investment counterparts made by FIs, were relevant for allowing the continuity of the processes started within RFI.

EVALUATION RECOMMENDATIONS

Based on the evaluation´s findings and conclusions, we propose 41 recommendations aimed at improving USAID´s future programming. The recommendations are grouped by evaluation question and listed below. The report´s main body includes the implementing measures defined for each recommendation, identifying the main stakeholders and implementation timeframe.

IMPROVEMENTS IN THE EFFICIENCY, EFFECTIVENESS AND QUALITY OF THE PORTFOLIO OF RURAL

FINANCIAL SERVICES

1. Continue combining TA and financial assistance targeted towards areas in which FIs require incentives to mobilize resources, while exploring alternative mechanisms to orient investments and activities to financial inclusion goals more efficiently. Support could be focalized towards mobilizing resources for serving specific population groups (women, youth, victims, ethnic minorities, migrants, among others).

2. Publish a compilation of the adjustments to internal processes that generated the best results for FIs within RFI, as innovative practices that others can replicate.

3. Develop and implement more systematized and adjusted risk assessment models for rural areas to speed-up credit approval and disbursement times.

4. Strengthen the VCF methodology, complementing it with better instruments to include business partners, business development services, and aligning the methodology with public policies currently being implemented by the government or other relevant actors.

5. Support Finagro in the design and implementation of requirements to regulate access to rediscount (second tier fund) resources by FIs that are not supervised by SFC nor Super Solidaria, based on the authorization included in Law 1955 of 2019.

6. Disseminate lessons learned from the implementation of the formerly designed DCAs3 within RFI.

7. Continue supporting Finagro in adjusting its guarantee policy, strengthening and expanding the Fondo Agropecuario de Garantías (FAG) to potentiate it as a sustainable instrument for financial inclusion.

8. Strengthen the follow-up and assistance processes offered through RFI to improve the credit portfolio quality and assuring this does not constitute a barrier for rural financial inclusion.

3 Note that RFI did not have an active role in DCA negotiations nor in its administration, as this was not part of the program contract.

9 | FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE USAID.GOV

9. Strengthen the incentives and TA offered to FIs to redesign savings products so that they adapt better to the requirements of rural clients, including reducing product cost and minimum savings amounts, granting savings rewards, among others.

10. Strengthen the incentives and TA offered to FIs for the design and implementation of new and improved rural insurance products, with an emphasis on voluntary insurance, damage insurance, agricultural insurance and parametric insurance, as well as on their promotion through digital channels.

11. Support the design and implementation of cross-bank interoperation strategies, seeking to make cross-bank transactions and open banking easier and less costly for both clients and FIs.

12. Publish a document compiling success cases of digital rural financial inclusion within RFI, as a strategy to promote the initiative and communicate best practices that others can replicate.

13. Support the design and implementation of mass communication campaigns highlighting the ease, low cost and reliability (security) of mobile money, with an emphasis on older customers.

14. Carry out a study characterizing digital culture barriers in rural areas, as an input for designing digital strategies and products that are adequate for this population.

15. Publish a document compiling best practices of client service in rural areas, including effective measures to reduce waiting time and mobilization costs, as well as highlighting the persisting needs regarding clarity and diffusion of information about financial products.

INCREASED ACCESS TO APPROPRIATE FINANCE SERVICES

16. Develop and implement mechanisms to expand the potential of BCs for rural financial inclusion, including expanding the quotas they can manage, and allowing them to carry out operations beyond transactions.

17. Continue supporting the joint work of USAID and BdO in the design and implementation of digital and mobile correspondent models according to what was established in the CONPES of Financial Inclusion, incorporating the lessons learned within RFI regarding rural BCs, mobile advisors and digital banking.

18. Support the mass use of digital banking channels for marketing, management and approval of financial products (including reaching clients through WhatsApp, approving credits and insurance online) that reduce attention times and operation costs in rural areas.

19. Continue supporting the provision of digital financial education initiatives

20. Incentivize the use of cell phones and data plans to increase rural population´s access to financial services

21. Develop strategies and offer TA to strengthen FIs focus on allocating first-time credits (for instance, building credit history, designing credit products for youth, articulating GLACs to the provision of formal credit).

USAID.GOV FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE | 10

22. Strengthen the technical and financial assistance offered to FIs on financial education, placing it as a strategy to build trust with rural clients, gather information on rural clients' needs and build better financial habits that can improve the quality of the loan portfolio.

23. Publish a document compiling best practices and recommendations for serving women and vulnerable populations with financial services in rural areas.

REDUCTIONS IN THE COSTS OF RURAL FINANCIAL SERVICES

24. Support large-scale digitalization strategies of processes that reduce costs and simplify the internal operation of FIs in rural areas.

25. Support the design and implementation of mass communication campaigns that explain rural clients the costs associated with different financial products, as a strategy to improve available information and trust towards FIs.

UNDERLYING VARIABLES MITIGATING OR ENHANCING THE IMPACT OF THE INITIATIVE

26. Align future financial inclusion programs with initiatives currently working on improving connectivity and digital abilities in rural areas, for instance, to focalize the initiative towards areas in which connectivity is being improved.

EFFECTS OF THE COVID-19 PANDEMIC

27. Continue supporting FIs in the creation of contingency plans to better respond to external shocks including natural disasters, economic crisis, pandemics, etc.

28. Support the design and implementation of strategies to allow FIs to keep active the rural clients that accessed the financial system for the first time during the pandemic (in order to receive a subsidy, preferential credits and other benefits provided by the Government).

IMPLEMENTATION STRATEGY AND ITS CONTRIBUTION TO THE ACTIVITY GOALS

29. Continue offering customized TA to each FI, while defining mechanisms that allow it to be even more adapted to the specific needs of each.

30. Revise and precise the definition of new client in future USAID Colombia financial inclusion programs and its indicators, so that financial inclusion results (first time access to the financial sector) can be more accurately determined.

31. Identify the most cost-effective measures to include rural clients in the financial system, through both traditional and non-traditional channels (especially digital ones), and based on this, define parameters and incentives to orient future RFI grants towards these.

32. If future programs include grants, breakdown their goals and investment lines by financial product (credit, savings, insurance, transactions) and areas of work (financial inclusion, differential approach, technological innovation, communications).

11 | FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE USAID.GOV

33. Introduce in the design and follow-up of grants, parameters that allow to standardize to a certain point, typologies of activities, minimum goals associated with each, and maximum costs associated with each (e.g., max financial incentive granted per new client in a certain region). This will allow a better comparison of the results of grants, and the identification of the activities that generate impacts in the most cost-effective manners (albeit acknowledging the natural differences that arise from different attention models, size and structures of FIs, and from differences in the characteristics of territories and targeted populations).

34. If future programs include grants, establish goals for new clients in every Incentive Grant, breaking these down by gender, youth and small producers or small enterprises.

35. Identify and accompany regulatory changes that could be a barrier for the sustainability or scaling-up of financial inclusion results (e.g., regulation on rural BCs, low value payment systems, subsidized interest rates).

36. Incentivize and accompany alliances between FIs, social organizations and local governments as a strategy to build trust, gather relevant information, implement GLACS and implement FE programs.

37. Complement RFI´s strategy to document and communicate best practices with ex post guides on best practices and other knowledge management products and compile them in the RFI web page, which should serve as a public access site.

38. Develop and support the implementation of specific attention models for rural women and vulnerable populations, aimed at changing the perspectives of FIs (especially local branches) so that they can better exploit the commercial potential of these population segments.

LESSONS LEARNED FROM THE ORIGINAL DESIGN

39. Continue with the one-on-one approach of working with FIs, but support and strengthen communication capacity within them (across the national and local offices and across different work areas) to ensure a more efficient and cross-cutting work within each FI.

40. If future programs include grants, require FIs to contribute with part of their own resources to all the activities included in their work plans, in order to align incentives, strengthen appropriation and increase the likelihood of sustaining the required investments.

41. Carry out ex post follow-up reports after the work plans have finished and analyze to what extent the activities have continued, been scaled-up or replicated.

USAID.GOV FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE | 12

EVALUATION PURPOSE4

Impact and performance evaluations aim at facilitating informed program management decisions and shaping the long-term strategic design and implementation of programs and decision making, contributing to USAID’s knowledge base to improve development outcomes. Evaluations support USAID/Colombia and the RFI implementing partner to account for results, provide lessons learned and insightful recommendations to strengthen programming decisions for future similar activities.

As stated in the Statement of Work (SoW), the purpose of this evaluation is to assess whether the RFI’s interventions, deliverables and outcomes are being accomplished according to the work plan and theory of change. The evaluation will identify to what extent the interventions, deliverables and outcomes are relevant based on the current national and local contexts. The scope of the evaluation is not limited to the implementation elements in the hands of the implementing partner, but also addresses design and learning elements directly related to USAID. Two types of evaluations were carried out:

The Impact Evaluation’s main methodological and design elements were defined in the RFI Baseline Impact evaluation (published in February 2018). The baseline data was collected in ten municipalities of the country (five treatments and five controls) and is taken as the starting point for the follow-up conducted in 2021. We compare data between both periods and municipality groups in a quasi-experimental causal analysis of RFI´s effects.

Performance Evaluation: The performance evaluation uses the qualitative information gathered in a broader sample of municipalities where the activity operated, as well as secondary information from multiple sources. This makes it possible to understand the results of the impact evaluation, identifying potential mechanisms in a broader spectrum of municipalities. Of equal importance to the impact evaluation, the performance evaluation identifies challenges, effective practices, lessons learned, and recommendations to improve management and implementation of the RFI activity. The performance evaluation supports learning and adapting by the implementing partner, informs key Colombian stakeholders and enables USAID to make informed decisions about the design and management of upcoming programming decisions.

Additionally, as part of the quantitative information collection, a follow-up of the Rural Financial Inclusion Survey for three municipalities in southern Tolima was included. The baseline was collected during 2016 in Rioblanco, Planadas and Ataco.5 This information is not used to answer the evaluation questions, but to expand knowledge and learning regarding financial inclusion, especially in these three municipalities. This analysis is written as an independent document.

4 We based this section on the Statement of Work for the Impact and Performance Evaluation of USAID/Colombia RFI.

5 This survey served as the main information source for the book, Inclusión financiera rural: el caso del sur del Tolima. Carlos Gustavo Cano, Paola Cuadros y Dairo Estrada; Bogotá. Banco de la República 2017.

13 | FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE USAID.GOV

ACTIVITY DESCRIPTION AND BACKGROUND

While Colombia has a relatively sophisticated financial system, and despite the country´s advances in social and economic development, challenges persist for the financial inclusion of millions of rural households. According to the latest agrarian census (2013), of the approximately 4 million rural producers in the country, only 11 percent accessed credit in the last 12 months. Formal savings and access to insurance are also uncommon in rural areas, and most money transfers and payments are still made in cash. Financial exclusion is especially high for rural households relative to small businesses (grocery stores, input stores, bakeries, mini-markets, etc.).6

Barriers for rural financial inclusion include, as documented by different studies7: i) systematic risk of the agricultural sector (associated with macroeconomic and climate volatility); ii) limited collateral; iii) high cost of client acquisition (associated with rural dispersion and poor physical and IT infrastructure); iv) low levels of financial literacy; v) insecurity associated with the armed conflict and illicit activities; and vi) low levels of productivity (associated with scarce access to technical assistance and public goods and to poor infrastructure).

On the other hand, among the factors that can facilitate rural finance, it is worth mentioning: i) helping financial institutions identify bankable opportunities; ii) strengthening value chain approaches to finance;

iii) designing adequate credit guarantees; iv) adapting financial products, services, and processes to rural conditions; v) improving rural producers’ financial literacy; and vi) relying on technology to reduce operating costs and access costs. To enhance these opportunities, different actors, including cooperation agencies such as USAID, have developed innovative approaches; one of these is the RFI.

The RFI supports the provision of market-based rural financial services for micro, small and medium-sized producers and businesses, including women, Afro-Colombians, indigenous, and other marginalized populations. The Initiative provides specialized technical and financial assistance to strengthen the capacity of participating FIs to better service rural entrepreneurs with client-responsive financial products. This is done through four components: i) Improved Rural Financial Intermediation; ii) Reduced Barriers to Rural Financial Services; iii) Modernized Financial Environment; and iv) Learning, Knowledge Management and Communications. Table 1 contains further information on the RFI activity, while Exhibit 1 presents the RFIs causal model8, explaining the expected development outcomes and the activities/components leading to each.

TABLE 1. RURAL FINANCE INITIATIVE INFORMATION

Activity Name: Rural Finance Initiative – RFI

Implementing Partner: Chemonics International

Contract Number: AID-514-C-15-00002

Total Estimated Cost: $22,670,554

6 See Assessment of Rural and Agricultural Finance and Financial Services in Colombia, UT Econometria – Marulanda Consultores, July, 2014.

7 Access to Financial Services in Rural Areas Market Study, Diana Parra Correa, OLGOONIK, July 7, 2014; Assessment of Rural and Agricultural Finance and Financial Services in Colombia, UT Econometria – Marulanda Consultores, July, 2014.

8 As defined in the Statement of Work and the Activity, Monitoring, Evaluation, and Learning Plan (AMELP).

USAID.GOV FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE | 14

TABLE 1. RURAL FINANCE INITIATIVE INFORMATION

Life of Project: August 3, 2015 to August 2, 2020

Active Geographic Regions:

The RFI activity is implemented in 197 municipalities in seven geographic corridors:

Bajo Cauca and Sur de Cordoba, Caquetá, Cauca, Meta, Montes de Maria, Sur de Tolima, Valle del Cauca, and Tumaco.

Alignment to the USAID/Colombia

CDCS 2014-2018:

DO 3: “Improved Conditions for Inclusive Rural Economic Growth”.

Intermediate Results: IR 3.2: “Increased Public and Private Investment in the Rural Sector.

External or internal Evaluation: External evaluation

Exhibit 1: RFI Causal Model

SOURCE: ACTIVITY MONITORING, EVALUATION AND LEARNING PLAN (AMELP)

The RFI contract was designed for RFI to work directly with and through FIs, to incentivize and build their capacity to deliver services to rural customers in target municipalities. Therefore, RFI, for the most part, did not work directly with the end beneficiary. Instead, FIs were the main recipients of RFI’s direct interventions to facilitate their capacity to provide services targeted at rural clients.

15 | FINAL EVALUATION REPORT – RURAL FINANCES INITIATIVE USAID.GOV

EVALUATION QUESTIONS AND METHODOLOGY

EVALUATION QUESTIONS

The evaluation questions go beyond the implementation of the RFI activity and reach topics related to the design, theory of change, context factors, among others, as described below:

IMPACT EVALUATION QUESTIONS:

1. To what extent has the RFI improved the efficiency, effectiveness and quality of the portfolio of financial services provided to municipal inhabitants?

2. How have the rural and vulnerable populations’ access to appropriate finance services changed as a result of the Initiative’s intervention?

3. To what extent has the Initiative reduced the costs of financial services for both banks and customers, and what innovative strategies (if any) has the RFI activity used to achieve these cost reductions?

4. What underlying variables potentially mitigate or enhance the impact of the RFI activity on the inhabitants of the target municipalities?

PERFORMANCE EVALUATION QUESTIONS:

1. To what extent has the RFI activity improved the efficiency, effectiveness, and quality of the portfolio of financial services provided to municipal inhabitants?

2. To what extent has the RFI activity reduced the costs of financial services for both banks and customers, and what innovative strategies (if any) has the Initiative…