ATTACHEMENT J.4 – DEC REPORTS USAID Ukraine AGRO Activity.pdf

AGRICULTURE GROWING RURAL

OPPORTUNITIES (AGRO) ACTIVITY

IN UKRAINE

MONITORING, EVALUATION, AND LEARNING PLAN

Award No: 72012120CA00001

Prepared for USAID/Ukraine C/O American Embassy 4 Igor Sikorsky St.

Kyiv 04112 Ukraine

Prepared by Chemonics International Inc.

1717 H Street NW Washington, DC 20006

Date of Original MEL Plan: February 13, 2020 Date of Last Revision: November 10, 2020

DISCLAIMER

This publication was produced by Chemonics International for review by the United States Agency for International Development. The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government.

ATTACHEMENT J.4 – DEC REPORTS USAID/Ukraine AGRO Activity i

TABLE OF CONTENTS

ABBREVIATIONS AND ACRONYMS ................................................................................................................ ii

MSME DEFINITIONS ..................................................................................................................................... iii

ACTIVITY DESCRIPTION

High-Level Results

Development Hypothesis

Activity Results Framework

COLLABORATING, LEARNING, AND ADAPTING PLAN

CLA Plan Matrix

CLA Team Structure

MONITORING AND EVALUATION

Precautionary Principle

Data Collection

Analysis

Data Management

Reporting

Data Quality Assessment Plan and Procedures

ANNEX I: CALENDAR OF TASKS

ANNEX II: PERFORMANCE INDICATOR SUMMARY TABLE

ANNEX III: DATA SOURCES AND INDICATORS TO BE COLLECTED FROM EACH SOURCE

ANNEX IV: PERFORMANCE INDICATOR REFERENCE SHEETS (PIRS)

ii

ABBREVIATIONS AND ACRONYMS

ADS Automated Directives System AGRO USAID’s Agriculture Growing Rural Opportunities Activity in Ukraine AOR Agreement Officer’s Representative ARDS USAID’s Agriculture and Rural Development Support Project CBO Community-based Organization CC Consolidated Community (Ukrainian “об’єднана територіальна громада,” ОТГ) CDCS Country Development Cooperation Strategy COP Chief of Party CLA Collaborating, Learning, and Adapting DCOP Deputy Chief of Party DO Development Objective DQA Data Quality Assessment ER Expected Result FTE Full-time Equivalent FY Fiscal Year GESI Gender and Social Inclusion GIS Geographic Information Systems

Ha hectare(s) IR Intermediate Result M&E Monitoring and Evaluation MEL Monitoring, Evaluation, and Learning Mln million MSMEs Micro, small and medium enterprises (includes farms) NGO Non-governmental organization OU USAID operating unit (for instance, a USAID Mission) PIRS Performance Indicator Reference Sheets PMP Performance Management Plan PMU Project management unit PPR Planning and Performance Report UAH Ukrainian Hryvnia USAID United States Agency for International Development USD United States dollar (US$) USG United States Government iii

MSME DEFINITIONS

Enterprise Category

Number of Employees

Annual Turnover, UAH

Annual Balance Sheet Assets, UAH

Micro <10 <1 million <500,000 Small <50 <15 million <5 million Medium <250 <100 million <15 million

Number of employees refers to the average number of workers during the reporting year.

MSMEs include producers (farmers). Producers should be classified as micro, small or medium-enterprises based on the number of FTE workers hired (permanent and/or seasonal) during the previous 12 months. If a producer does not hire any permanent or seasonal labor, s/he should be considered a micro-enterprise.

“Household farms” are small farms or large gardens, often directly around the house, run by a family and producing mostly for autoconsumption even though they may sell some of their produce. Ukrainian law defines them as noncommercial farms. AGRO generally considers them to be “microenterprises.”

“Family (peasant) farms” are a specific category of commercial farm defined in Ukrainian law that produce for market sale even though they may be family-owned and operated. They may be micro-, small or medium enterprises depending on how they fit the criteria above.

ACTIVITY DESCRIPTION

USAID’s Agriculture Growing Rural Opportunities (AGRO) Activity in Ukraine’s goal is to accelerate the economic development of rural Ukrainian communities with the greatest need through a better governed agricultural sector that encourages more productive, modern, and profitable micro, small, and medium agricultural enterprises (MSMEs) that are successfully integrated into competitive markets both in Ukraine and internationally.

The AGRO Activity comes at a critical time for Ukraine, when ensuring forward movement on enabling environment reforms and expanding agriculture sector gains are vital for economic growth and development. AGRO will harness and build on national and grassroots momentum in enabling environmental reform and scalable market system solutions primarily in the East and South to achieve accelerated economic development in rural communities with the greatest need.

High-Level Results

The high-level results for AGRO are:

• Increased revenue of agricultural small and medium enterprises (SMEs)

• Increased access for agricultural SMEs to financing

• Establishment of a fair and transparent agricultural land market (land reform)

• Reducing corruption in the agricultural sector

Development Hypothesis

The development hypothesis of the AGRO activity is:

IF AGRO encourages more productive, modern, and profitable small and medium agricultural businesses that are successfully integrated into competitive markets both in Ukraine and internationally through improved value chains and supporting functions AND improves the governance of the agricultural sector of Ukraine; THEN economic development of rural Ukrainian communities will be accelerated, supporting Ukraine’s Journey to Self-Reliance.

Activity Results Framework

This activity level results framework demonstrates AGRO’s links to the Mission’s results framework as well as the logical, causal relationships between the different levels of results.

COLLABORATING, LEARNING, AND ADAPTING PLAN

The AGRO Monitoring, Evaluation and Learning (MEL) system informs the Activity’s collaborating, learning, and adapting (CLA) approach to planning, implementation and evaluation. CLA allows for a high degree of flexibility to respond to changes in Ukraine’s operating environment, adapting lessons learned and ensuring successful Activity implementation. The CLA approach enables AGRO to understand how program implementation is affecting our partners and agricultural business environment and adjust accordingly.

As part of AGRO’s philosophy, CLA aims to:

• Improve the quality and relevance of all activities by grounding them in evidence;

• Make activities adaptive to new learning and changing contexts;

• Extend USAID influence through sharing knowledge and collaboration with development actors; and

• Catalyze learning among Ukrainian development actors to build capacity and facilitate country-led development.

AGRO prioritizes collaboration, learning, and adapting with individuals engaged by the Activity and those working within the Activity. CLA can be divided into three inter-related areas, which refer to different target audiences or systems.

• Internal processes: Organizational and staff management practices that contribute to finding synergies within the project, such as efficient internal communications between AGRO teams, culture of open discussions, IT infrastructure, and investments in staff learning and skill-building.

• International development environment: Tools and processes used to communicate, collaborate, further share and learn with other stakeholders such as other development projects and donors.

• External processes: Technical implementation of AGRO, its relation to the market system and how its actors cooperate, learn, and adapt. This last area is often an increasingly important integrated aspect of facilitation activities.

Collaboration. AGRO will collaborate with USAID and other donor projects as well as other stakeholders to design, implement, monitor, learn from, adapt, and scale effective programming.

Collaboration and coordination in providing agricultural development assistance is critical, both because of the importance of agriculture within the Ukrainian economy and because of the large number of international actors supporting Ukraine within the sector.

AGRO will focus particularly on engagement with the World Bank. Strategy will be coordinated at the USAID level. The AGRO team will use coordination methods that will be adapted throughout the life of AGRO as appropriate:

i. Participation in expert groups;

ii. Regular coordination meetings;

iii. Joint implementation planning

iv. Frequent plan reviews and adaptation informed by the progress of reforms;

v. Joint facilitation of events and reform communications; and

vi. Sharing of experience with World Bank partners.

Collaboration also means AGRO engagement and co-creation of solutions with private sector partners, their supply chain partners, industry associations, consolidated communities, and government. Finally, AGRO will apply collaborative approaches as part of facilitation: AGRO will promote collaborative and strengthened relationships among value chain actors, between financial institutions and their clients, and between policymakers and the public.

Learning. AGRO must be proficient in capturing, systematizing, drawing lessons from, and applying explicit and tacit knowledge. Explicit knowledge is public, often written and published for all to access. Tacit knowledge emerges through observations and experience. As a result, ensuring tacit knowledge is captured and effectively used to monitor systemic change requires learning/sharing tactics that encourage information to flow more effectively. To catalyze more effective information flows AGRO expects, for example, to:

vii. Conduct special studies and analyses around learning questions;

viii. Hold annual pause-and-reflect events and on-the-spot learning reviews at mid-year to assess what is and is not working to revise the annual implementation plan;

ix. Organize regular staff exchanges between donor partners, other projects, team members, etc. to encourage new perspectives and insights

x. Organize learning events and activities for AGRO innovating and early adopter stakeholders;

xi. Hold learning events with USAID project

MEL and technical staff to identify learning priorities;

xii. Publish articles on project approaches and innovations. Published materials may focus on how AGRO intervention can be replicated.

xiii. Publish case studies, briefs, and tools documenting approaches tested by the Activity and the learning drawn from them.

Identifying best practices for private sector engagement and policy reform supports learning by USAID/Ukraine as part of its Performance Management Plan (PMP) and Country Development Cooperation Strategy (see box).

Lessons learned will be integrated into stakeholder consultations; ongoing and new approaches and interventions, to make course corrections and to scale up or down. They will also be intensively examined within the project, in everyday operations and at annual pause-and-reflect workshops at the mid-year point and as part of implementation planning.

Supporting USAID’s CDCS PMP Learning Questions

AGRO will provide data and information to support the mission-wide learning question, “How can we bring together the private sector, citizens, and the to work together organically”, as well as several DO4 learning questions, such as “To what extent is the portfolio creating positive feedback loops between reducing corruption and advancing market-based, inclusive economic growth?” and “To what extent are activities under the DO taking an inclusive approach to creating new economic opportunities?”

Adapting and adaptive management. An adaptive management model will be used for Activity implementation. Based on results and lessons learned during implementation with value chain actors, national and local government, and civil society, AGRO will continuously make necessary adjustments for future interventions and discontinue those that do not achieve the intended results. Given the fluid Ukrainian environment, AGRO’s adaptive management approach will rely on up-to-date MEL data; systemic change indicators at the intervention level;

situational and political analysis undertaken by the Activity, other donors, and Ukrainian thought leaders; and recent political economy analyses to identify opportunities for successful implementation and to minimize Activity uncertainties.

AGRO will also adaptively manage the program with USAID through such means as:

• Adaptive annual implementation plans to be approved by the Agreement Officer’s Representative. Implementation plans will serve as practical management tools for the AGRO team and USAID. The annual plan will cover 12 months of detailed activities, plans and milestones; updates and changes, if any, will be formally reviewed and approved by the AOR every six months.

• Formal quarterly check-ins to inform adaptive management decisions. Quarterly check-ins will include a high-level review of Activity workstreams, progress, and challenges and discussion of immediate or upcoming decision points for the Activity. Quarterly check-ins will also incorporate discussion of the budget execution progress.

• On-demand informal activity reviews as significant program activities begin or close.

• Check-ins and AOR notification if significant changes to the Activity scope or plans occur or are anticipated.

CLA Team Structure

Chief of Party. The Chief of Party (COP) is ultimately responsible for the activity including implementation of the CLA activities just described and this MEL plan. The COP supervises all collaborative activities and reviews and approves all reports that include data and findings, as well as leading the staff in interpreting data for learning and activity adaptation. The COP supervises the Collaborating, Learning, and Adapting Team. The CoP, through the DCOP and the CLA Team, ensures that all subcontractors or sub-awardees and the Activity team are working cohesively in collecting, analyzing, using, and disseminating project data and learning.

The COP, DCOP and the CLA team review activity indicators twice each year, to ensure that they continue to be relevant and well-formulated for management, communication, learning, and adapting and if appropriate, propose changes in them to the AOR.

CLA Advisor. The CLA advisor leads CLA and knowledge management activities by providing tools and training for capturing and sharing lessons learned, innovations, and best practices, and ensuring that such learning is adequately documented and disseminated. The Advisor ensures that CLA is integrated throughout AGRO activity design and implementation as a crosscutting factor, oversees GESI, MEL and internal and external communications activities, and manages related technical activities implemented by contractors and/or sub-awardees.

MEL managers. The MEL managers have primary responsibility for overseeing the identification, collection, and analysis of MEL data. The MEL managers lead the activity staff, implementing partners, and any local firm in developing the necessary tools and systems for quality data collection. With the CLA Advisor, the MEL managers design and conduct training on the meaning and interpretation of the MEL plan, its indicators, data collection tools and systems. They have primary responsibility for the quality of data entered into the AGRO M&E system. They check documentation and data records, maintain geolocation data for all activity sites, and perform an annual data quality assessment (DQA).

Gender and Social Inclusion Specialist. The GESI specialist ensures that awareness of gender and social inclusion is maintained throughout AGRO. The GESI Specialist is responsible for the development of AGRO’s GESI strategy. With the CoP, DCoP, CLA Advisor, and technical team leads, the GESI Specialist ensures the incorporation and implementation of the GESI strategy and that inclusion targets are met throughout all AGRO activities. The Specialist organizes the production of relevant materials focused on inclusion and conducts training on gender and social inclusion for staff, partners and other stakeholders as needed and appropriate.

Technical staff. Data collection is a team effort and is built into the responsibilities of all technical staff. Technical team members are responsible for front-line data collection within their respective activities. They conduct field visits wherever in Ukraine AGRO has an activity to monitor implementation and ensure accurate data collection and reporting. Technical staff share performance data and engage in discussions about data and findings with AGRO’s local partners.

As appropriate and directed by the CoP, they also liaise with , business and civil society organizations and with other USAID- and donor-supported projects.

AGRO contractors, sub-awardees, and partners. AGRO engages contractors, sub-awardees, and partners in data collection, analysis, and use as required. This approach supports partners’ organizational development as well as AGRO’s monitoring and reporting efforts. Grant agreements (and subcontracts where appropriate) will include expectations, responsibilities and deliverables for monitoring and evaluation. Grantees will be consulted in the process of finalizing baselines and targets as appropriate and will be trained in data collection, analysis, and review.

MONITORING AND EVALUATION

Effective monitoring, evaluation, and learning requires that activity results and indicators be set early and tracked and analyzed consistently to determine whether goals are being met and, if not, to provide sufficient information to understand why not and correct the course of action. The purpose of this MEL plan is not simply to organize the collection of data, but to guide accurate assessment of AGRO’s impact. The AGRO activity team will use data collected to determine whether the activity is meeting its performance requirements, to inform decision making, and to improve activity management and implementation. This MEL plan describes AGRO’s methodology for activity monitoring and evaluation and will be used to plan and manage the collection of performance data and measure the success of the activity in advancing economic development of Ukrainian rural communities. The MEL plan results will be used to regularly update and engage relevant stakeholders, including USAID, to analyze change and adapt to results.

Annex I summarizes, assigns responsibility for, and schedules the tasks required to carry out this MEL plan. The indicator table in Annex II summarizes performance indicators, targets, data sources and collection methods, frequency of data collection, disaggregation, and units of measure. Annex III tabulates data sources and when they will be queried for information, and Annex IV is the detailed Performance Indicator Reference Sheets (PIRS).

The CLA team and the activity team will revisit this MEL plan annually to review its appropriateness and adapt indicators and targets as needed. The MEL plan is a living document and necessary modifications can be made at any time. All changes will be well documented and recorded in AGRO M&E files.

All of AGRO’s operations, including its MEL activities, will fully comply with the Law of Ukraine “On Personal Data Protection.”

Precautionary Principle

The principle “First, do no harm” is central to AGRO’s data collection, analysis, and dissemination. We understand that how key terms are defined, who collects data and from whom, how it is collected, how data is analyzed and interpreted, and who does and does not get to see findings from monitoring and evaluation can inflame existing social and political tensions, reinforce social exclusion and oppression, and do real emotional and physical harm. This MEL carefully considers potential harm and its implementation will always be sensitive to possible harm from M&E activities.

Data Collection

Indicators. AGRO’s indicators are listed in Annex II. Most indicators are custom ones. The indicators were developed from a review of indicators in the cooperative agreement, USAID/Ukraine’s Country Development Cooperation Strategy 2019-2024, USAID’s Feed the Future Indicators, USAID’s Gender Equality and Female Empowerment Policy, and Standard F Indicators. The review identified a set of indicators that are direct, objective, practical, adequate, and useful in managing for results. Indicators are at the output level that measure activity interventions and at the outcome level that measure the results of those interventions. This two-tiered approach ensures that both activity process and activity design are measured, allowing assessment of the validity of the activity’s development hypothesis and faithfulness to the original activity design.

Three standard indicators from the United States Government Foreign Assistance Framework are included:

• USD sales of firms receiving USG-funded assistance (EG. 5.1);

• Percentage of female participants in USG-assisted programs designed to increase access to productive economic resources (assets, credit, income or employment) (GNDR-2); and

• Percent of USG-assisted organizations with improved performance (CBLD-9).

CBLD-9 is a standard indicator that the USAID Mission was mandated to track, as USAID is currently rolling out the New Partnership Initiative (NPI) and applying the new Agency-wide approach to measuring improved performance of new and underutilized partners through developing their organizational capacity.

At a later point, AGRO may add an indicator on the “Value of agricultural and rural loans as a result of USG assistance,” to be tracked as an additional disaggregation of the indicator “Number of supported MSMEs accessing finance.”

For indicators that are given in percent, AGRO will maintain records of the absolute values from which the percentages were calculated and report those underlying values on request.

Disaggregation of data. All person-level indicators are disaggregated by biological sex. Other disaggregations include location, value chain, business size, and type of technology or management practice as appropriate to specific indicators. The disaggregated data provide detailed information that allow examination of the effectiveness of activity interventions and tighter focusing of interventions to reach desired audiences.

Baselines and Targets. Following guidance received from the Mission Kyiv M&E specialist, AGRO baselines reflect an indicator’s status before major USAID intervention in the area the indicator measures. So most AGRO baselines repeat or, if needed because of changes in the indicator, slightly modify baseline values used for the previous USAID-funded ARDS Project.

To set annual targets, local experts, stakeholders, ARDS champion firms, and partners were consulted, and the sequencing and timing of activities carefully considered.

ARDS and AGRO overlap from November 15, 2019 through July 31, 2020. As the projects transition, assigning a result to one activity or the other is often arbitrary. Therefore, in some cases AGRO will report the same indicator values as ARDS, double-counting them. AGRO quarterly reports will clearly state when this is the case.

Data sources and collection methods. When possible, AGRO will report data from generally-used secondary sources, In many instances, however, the activity will collect its own primary data. Annex III lists the expected provider of data for each indicator.

A significant portion of AGRO’s data needs can be fulfilled through existing data sources, either internal or external records (primary and secondary sources). Internal records include grant and technical implementation (including site visit) documents. The program’s focus on MSMEs as beneficiaries and drivers of economic growth, however, requires that significant data be collected from them directly.

The CoP, CLA Advisor and MEL managers will oversee the design and wording of all survey, focus group, and structured interview questions. Survey instruments will be pretested.

Multi-tier approach. The strength of AGRO’s monitoring is in its multi-tiered approach to collecting, verifying, and reporting data. This approach enables AGRO to manage results and to continually improve activity performance.

• Tier 1: Data that project staff, subcontractors, and grantees collect directly

• Tier 2: Relevant data from other USG or other donor activities

• Tier 3: Data from the host country government and local civil society organizations

• Tier 4: Local monitoring firms and organizations

Monitoring information will, as much as possible, be collected with and through partner organizations, building capacity and ensuring stakeholder buy-in from the earliest stages of MEL planning and development, to data analysis and learning.

Analysis

The CoP, DCoP, CLA team and technical teams will all contribute to data analysis. The CLA Advisor and MEL managers will ensure that valid, reliable data is available and produce statistical reports for the AGRO team. The technical teams’ knowledge will further understanding and interpretation of the data in light of their knowledge of the interventions, that AGRO has implemented and the challenges those interventions have faced.

Learning. As already discussed, AGRO’s CLA approach allows for a high degree of flexibility to respond to changes in the operating environment, adapting lessons learned to ensure successful implementation. To ensure learning from MEL data, the AGRO team has developed a CLA matrix (please see above) that outlines when the data will be reviewed, who will be present, and how to capture and disseminate the findings. Each quarter, the team will begin with the project data from the previous quarter, analyzing findings, raising issues and noting trends. Together, the staff will identify what information beyond that required for the MEL plan indicators will be the most helpful in the coming quarter, what resources exist to obtain this information, and how to capture knowledge once generated. AGRO will add at least one policy analyst and an agricultural economist to the staff shortly after inception, and those specialists’ knowledge will also be drawn on to foresee challenges and adapt to them.

Perverse incentives. The learning plan ensures that indicators and data collection methods do not create incentives for behaviors that are unproductive, unethical, or problematic for beneficiaries, AGRO or other stakeholders. AGRO technical and CLA team members will hold regular discussions with regional partners on whether and what perverse incentives could be arising due to program activity. At annual implementation planning in conjunction with key stakeholders, a portion of the agenda will consider what perverse incentives then exist and how to address them.

Unintended consequences. Careful attention will be paid to unintended consequences of AGRO interventions. Positive and negative unintended consequences will be reviewed, along with strategies to mitigate negative consequences and share positive consequences.

Data Management

Chemonics uses DevResults to manage M&E data and to measure performance against targets for each of its projects. DevResults software is specifically designed for use by international development programs’ M&E and can generate maps, charts, and graphs. It also supports data collection, reporting, task management, and selected financial tracking activities and has GIS, document, and photo storage capabilities. The software allows various users, including USAID, to access data and results in real time, contributing to our principle of transparency. Chemonics provides home office support for project compliance with ADS 579, particularly preparation of datasets for submission to USAID’s Data Development Library. Datasets will be submitted in a nonproprietary format to the USAID AOR or elsewhere as directed.

Reporting

AGRO will provide USAID with MEL data updates in quarterly and annual reports, including the Activity final report. These reports will present a table of indicator values and a narrative of progress toward each expected result. Annual and quarterly reports will illustrate progress toward targets, challenges, strategies for overcoming challenges, and key successes. User-friendly formats will highlight information. Quantitative data may be represented via infographics, such as dashboards, charts, bars, and graphs. Quantitative data will be supplemented with narratives to clarify the values’ context.

AGRO will regularly provide USAID with success stories and other communication materials that will blend data with personalized stories to personify and illustrate the program’s activities, outcomes, and overall impact.

AGRO MEL data will be used not only for reporting but also to inform project information, education, communication and outreach, such as publications and media campaigns.

Data Quality Assessment Plan and Procedures

USAID requires that performance indicator data reported externally, including annual Planning and Performance Reports (PPR) sent to USAID/Washington, must have a data quality assessment (DQA), to ensure that managers are aware of the strengths and weaknesses of the data and the extent to which they can be trusted in shaping management decisions. A DQA must be conducted at least 12 months before reporting data to USAID for new indicators and every three years thereafter. USAID will provide AGRO with DQA checklists for completion that will then be reviewed and signed by the AOR.

In conjunction with the home office, the CLA team will lead annual internal data quality assessments using Chemonics’ established data quality assessment guidelines, instruments and procedures. The person responsible for collecting the data for a particular indicator will not be responsible for the DQA of that indicator.

ANNEX I: CALENDAR OF TASKS

MEL Tasks Responsible party

FY2020 FY2021 FY2022 FY2023 FY2024

Notes

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

Develop the MEL Plan and conduct annual reviews and updates

CoP / DCoP / CLA Advisor / MEL Manager

X

X X X X

The MEL plan will be reviewed annually with updates made as necessary.

Design and update M&E and CLA data collection and reporting tools

/ CLA Advisor

X X X X X

Document and collect activity data as per implementation

MEL Manager / Technical team members

X X X X X X X X X X X X X X X X X X

Done on an ongoing basis

Design and upgrade the AGRO database

X X X X X X X X

The database will be updated bi-annually

Aggregate and compile indicator tracking data

MEL Manager

X X X X X X X X X X X X X X X X X X

To be done quarterly, bi-annually, and annually depending on the indicator

Orient the AGRO team on data collection and reporting tools

FY2020 FY2021 FY2022 FY2023 FY2024

Notes

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

Conduct M&E training for the Project team and partners

MEL Manager X X X X X

This is formal, structured training

Provide ongoing M&E technical support to the Activity staff

Conduct data quality assessments

For indicators reported externally, DQAs will be conducted in accordance with USAID checklist.

Perform data cleaning MEL Manager X X X X X X X X X X X X X X X X X X To be done by the M&E team

Hold quarterly, bi-annual, and annual performance reviews

CLA Advisor X X X X X X X X X X X X X X X X X

Conduct data analysis MEL Manager

/ Technical team members as appropriate

Produce quarterly, bi-annual, and annual reports for USAID

MEL Manager / CLA Advisor / CoP / DCoP

Disseminate reports, success stories, best practices, and lessons

X X X X X X X X X X X X X X X X

FY2020 FY2021 FY2022 FY2023 FY2024

Notes

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 learned to key stakeholders

Design, organize and manage surveys

MEL Manager X X X X X

Design, organize and manage focus groups

MEL Manager As needed

Conduct end of Activity evaluation

/ CLA Advisor X

ANNEX II: PERFORMANCE INDICATOR SUMMARY TABLE

PERFORMANCE

INDICATOR

INDICATOR

DEFINITION

DATA

SOURCE

BASELINE

DATA

TARGETS AND ACTUALS

FY 2020 FY 2021 FY 2022 FY 2023 FY 2024 Life of Project Target

Year Value Target Actual Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

Goal: Economic development of rural Ukrainian communities with the greatest need accelerated through a better governed agricultural sector, to encourage more productive, modern, and profitable small and medium agricultural businesses that are successfully integrated into competitive markets both in Ukraine and internationally

1. Indicator Name: Value in UAH of new private sector investment in USAID-assisted agricultural MSMEs

Unit of Measure: million

UAH

PIRS

Investment is any use of private sector resources intended to increase future production, output, or income, improve the sustainable use of agriculture-related natural resources (soil, water, etc.), and/or improve water or land management, etc.

AGRO Activity and partner financial records

2017 UAH

UAH 123 UAH

UAH 245 UAH 306 UAH 306 UAH 245 UAH

1,226

2. Indicator Name:

Number of new USAID-assisted rural investment projects implemented that benefit agricultural MSMEs

Unit of Measure: Number

PIRS

Number of new rural investment projects financed as a result of USG assistance.

AGRO

Activity, partner and municipal government records

2019 15 0 0 10 16 16 6 48

Objective 1: Productivity and Market Access in Target Value Chains Improved

INDICATOR

INDICATOR

DEFINITION

DATA

SOURCE

BASELINE

DATA

TARGETS AND ACTUALS

FY 2020 FY 2021 FY 2022 FY 2023 FY 2024 Life of Project Target

Year Value Target Actual Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

3. Indicator Name:

Percentage of productivity change among agricultural MSMEs as a result of USG assistance

Unit of Measure: Percent

PIRS

Productivity for each value chain will be calculated separately because the unit for measuring yields across different value chains varies.

AGRO Activity and partner financial records

2017 10% 5% 5% 8% 8% 5% 31%

Expected Result 1.1: Use of high-quality inputs and improved production practices facilitated

Expected Result 1.2: Infrastructure for post-harvest handling, storage, and processing improved

INDICATOR

DEFINITION

DATA

SOURCE

BASELINE

DATA

TARGETS AND ACTUALS

FY 2020 FY 2021 FY 2022 FY 2023 FY 2024 Life of Project Target

Year Value Target Actual Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

4. Indicator Name:

Number of farmers and others who have applied improved technologies or management practices with USG assistance

Unit of Measure: Number

PIRS

This indicator measures the total number of direct beneficiary farmers, ranchers and other primary sector producers, as well as individual processors (not firms), rural entrepreneurs, traders, natural resource managers that applied improved technologies anywhere within the food and fiber system as a result of USG assistance during the reporting year.

Annual survey on technologies, partner records, and site visit reports

2017 125 300 22

Female:1

Male:10

Fruit/Ve getables/ Berries:1

Milk/Me at:7

Confecti onary:3

500 500 500 500 2,300

Expected Result 1.3: Market Access of Ukrainian agricultural SMEs increased

INDICATOR

DEFINITION

DATA

SOURCE

BASELINE

DATA

TARGETS AND ACTUALS

FY 2020 FY 2021 FY 2022 FY 2023 FY 2024 Life of Project Target

Year Value Target Actual Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

5. Indicator Name: USD sales of firms receiving USG-funded assistance

(EG. 5.1)

“E” – export

“D” – domestic

Unit of Measure: million current USD

PIRS

USD sales is defined as the total annual revenue of assisted firms for the reporting period, typically for the last 12 months at the point of data collection. Include both domestic sales and exports.

AGRO Activity and partner financial records

2017 USD

USD

195D

USD

29E

USD 81

USD 77 D

USD 4 E

USD 204

USD 194D

USD 10 E

USD 347

511D

USD 13 E

USD 604

595D

USD 9 E

USD 652

646D

USD 6 E

USD

646 D

6 E

Objective 2: Key Supporting Functions Improved

6. Indicator Name:

Number of times USG-assisted service providers perform new or improved services for agricultural MSMEs.

Unit of Measure: Number

PIRS

Number of times MSMEs received new or improved services provided by market system actors (service providers) that had received support from

AGRO

AGRO Activity and partner financial records

2017 800 1,500 1,266 1,600 1,750 1,900 2,100 8,850

Expected Result 2.1: Access to efficient and affordable irrigation improved

INDICATOR

DEFINITION

DATA

SOURCE

BASELINE

DATA

TARGETS AND ACTUALS

FY 2020 FY 2021 FY 2022 FY 2023 FY 2024 Life of Project Target

Year Value Target Actual Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

7. Indicator Name:

Number of beneficiaries who have accessed improved irrigation systems

Unit of Measure: Number

PIRS

The number of beneficiaries who have accessed improved irrigation systems as a result of the USG support

Local communities, local government, water management agencies, and AGRO Activity records

2020 0 0 0 200 300 300 200 1,000

Expected Result 2.2: Access to finance increased

8. Indicator Name:

Number of supported MSMEs accessing finance

Unit of Measure: Number

PIRS

Total number of micro, small, and medium enterprises (MSMEs) that have received USG assistance accessing finance during the reporting year.

AGRO Activity and partner financial records

2017 4 100 177 350 550 500 400 1,900

Objective 3: Agricultural Sector Rules and Regulations Improved

INDICATOR

DEFINITION

DATA

SOURCE

BASELINE

DATA

TARGETS AND ACTUALS

FY 2020 FY 2021 FY 2022 FY 2023 FY 2024 Life of Project Target

Year Value Target Actual Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

9. Indicator Name:

Number of agriculture and rural development-related rules and regulations advanced with USG assistance

Unit of Measure: Number

PIRS

Number of specific pieces of legislation (laws, rules, regulations, procedures, etc.)

developed, adopted, and/or enforced with AGRO assistance that are related to agriculture and rural development including land reform.

AGRO Activity and government records

2017 9 5 5 15 15 10 10 55

Expected Result 3.1: National enabling environment improved

10. Indicator Name:

Percentage of agricultural MSMEs that report facing corruption in land relations

Unit of Measure: Percent

PIRS

The proportion of agricultural businesses of all sizes that report they encountered a corrupt practice in land relations.

Annual survey 2017 64% 33% 32% 31% 30% 29% 29%

Expected Result 3.2: Local enabling environment improved

INDICATOR

DEFINITION

DATA

SOURCE

BASELINE

DATA

TARGETS AND ACTUALS

FY 2020 FY 2021 FY 2022 FY 2023 FY 2024 Life of Project Target

Year Value Target Actual Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

11. Indicator Name:

Hectares of land opened for transparent public view with USG assistance

Unit of Measure: million hectares

PIRS

The area of land that has been mapped and plotted on publicly-accessible GIS portals as a result of the consolidated communities’ (CCs) implementation of the participatory land inventory/management methodology originally developed by ARDS.

This process will not begin on a large scale until after has passed draft law 2280, expected in the spring of 2020.

AGRO

Activity, partner and municipal government records

2019 0,4 0,03 0,047 0,57 0,6 0,6 0,7 2,5

INDICATOR

DEFINITION

DATA

SOURCE

BASELINE

DATA

TARGETS AND ACTUALS

FY 2020 FY 2021 FY 2022 FY 2023 FY 2024 Life of Project Target

Year Value Target Actual Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

12. Indicator Name:

Number of communities implementing improved land management techniques as a result of USG assistance

Unit of Measure: Number

PIRS

Number of CCs that implement the innovative, technology-based, participatory local land management model that has been developed with USG assistance. This process will not begin on a large scale until after has passed draft law 2280, expected in the spring of 2020.

AGRO

Activity, partner and municipal government records

2019 13 60 61 77 90 90 100 417

Crosscutting

13. Indicator Name:

Percentage of female participants in USG-assisted programs designed to increase access to productive economic resources (assets, credit, income or employment)

(GNDR-2)

Unit of Measure: Percent

PIRS

Productive economic resources include assets – land, housing, businesses, livestock or financial assets such as savings; credit; wage or self-employment;

and income.

AGRO Activity records, annual survey of MSMEs, government and partner records

2017 28%

N:129

D:464

45%

N:4,100

D:9,200

43%

N:1,933

D:4,523

47%

N:4,580

D:9,800

47%

N:5,280

D:10,600

50%

N:5,830

D:11,600

50%

N:6,430

D:12,800

50%

N:6,430

D:12800

INDICATOR

DEFINITION

DATA

SOURCE

BASELINE

DATA

TARGETS AND ACTUALS

FY 2020 FY 2021 FY 2022 FY 2023 FY 2024 Life of Project Target

Year Value Target Actual Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

Target

A c t u a l

14. Indicator Name:

Percentage of women who report improved economic well-being at the household level as a result of USG assistance

Unit of Measure: Percent

PIRS

This impact level indicator measures the percentage of women out of those involved in AGRO activities, who report that economic well-being of their households has improved as a result of AGRO’s interventions.

Annual survey 2019 0 10% 25%

N:17

D:68

10% 10% 10% 10% 10%

15. Indicator Name:

Percentage of USG-assisted organizations with improved performance

(CBLD-9)

Unit of Measure: Percent

PIRS

This indicator measures whether USG-funded capacity development efforts have led to improved organizational performance in organizations receiving organizational capacity development support.

Project records, annual survey of MSMEs, government and partner records.

2019 0 25%

N:138

D:550

64%

N:359

D:559

65%

N:715

D:1,100

70%

N:980

D:1,400

70%

N: 945

D:1,350

70%

N:735

D:1,050

70%

N:735

D:1,050

ANNEX III: DATA SOURCES AND INDICATORS TO BE COLLECTED FROM EACH

SOURCE

No. Indicator

Data Sources

Project records SMEs Surveys Partners records

1. Value in UAH of new private sector investment in USG-assisted agricultural MSMEs

2. Number of new USG-assisted rural investment projects implemented that benefit agricultural MSMEs

3. Percentage of productivity change among agricultural MSMEs as a result of USG assistance

4. Number of farmers and others who have applied improved technologies or management practices with USG assistance

5. USD sales of firms receiving USG-funded assistance (EG. 5.1)

6. Number of times USG-assisted service providers perform new or improved services for agricultural MSMEs.

7. Number of beneficiaries who have access to improved irrigation systems

8. Number of supported MSMEs accessing finance

9. Number of agriculture and rural development-related rules and regulations advanced with USG assistance

10. Percentage of agricultural MSMEs that report facing corruption in land relations

11. Hectares of land opened for transparent public view with USG assistance

12. Number of communities implementing improved land management techniques as a result of USG assistance

13.

Percentage of female participants in USG-assisted programs designed to increase access to productive economic resources (assets, credit, income or employment) (GNDR-2)

14. Percentage of women who report improved economic well-being at the household level as a result of USG assistance

15. Percentage of USG-assisted organizations with improved performance (CBLD-9)

IN UKRAINE

MONITORING, EVALUATION, AND LEARNING PLAN

Award No: 72012120CA00001

Prepared for USAID/Ukraine C/O American Embassy 4 Igor Sikorsky St.

Kyiv 04112 Ukraine

Prepared by Chemonics International Inc.

1717 H Street NW Washington, DC 20006

Date of Original MEL Plan: February 13, 2020 Date of Last Revision: November 10, 2020

DISCLAIMER

This publication was produced by Chemonics International for review by the United States Agency for International Development. The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United i

ABBREVIATIONS AND ACRONYMS ................................................................................................................ ii

MSME DEFINITIONS ..................................................................................................................................... iii

ACTIVITY DESCRIPTION

High-Level Results

Development Hypothesis

Activity Results Framework

COLLABORATING, LEARNING, AND ADAPTING PLAN

CLA Plan Matrix

CLA Team Structure

MONITORING AND EVALUATION

Precautionary Principle

Data Collection

Analysis

Data Management

Reporting

Data Quality Assessment Plan and Procedures

ANNEX I: CALENDAR OF TASKS

ANNEX II: PERFORMANCE INDICATOR SUMMARY TABLE

ANNEX III: DATA SOURCES AND INDICATORS TO BE COLLECTED FROM EACH SOURCE

ANNEX IV: PERFORMANCE INDICATOR REFERENCE SHEETS (PIRS)

ii

ABBREVIATIONS AND ACRONYMS

ADS Automated Directives System AGRO USAID’s Agriculture Growing Rural Opportunities Activity in Ukraine AOR Agreement Officer’s Representative ARDS USAID’s Agriculture and Rural Development Support Project CBO Community-based Organization CC Consolidated Community (Ukrainian “об’єднана територіальна громада,” ОТГ) CDCS Country Development Cooperation Strategy COP Chief of Party CLA Collaborating, Learning, and Adapting DCOP Deputy Chief of Party DO Development Objective DQA Data Quality Assessment ER Expected Result FTE Full-time Equivalent FY Fiscal Year GESI Gender and Social Inclusion GIS Geographic Information Systems

Ha hectare(s) IR Intermediate Result M&E Monitoring and Evaluation MEL Monitoring, Evaluation, and Learning Mln million MSMEs Micro, small and medium enterprises (includes farms) NGO Non-governmental organization OU USAID operating unit (for instance, a USAID Mission) PIRS Performance Indicator Reference Sheets PMP Performance Management Plan PMU Project management unit PPR Planning and Performance Report UAH Ukrainian Hryvnia USAID United States Agency for International Development USD United States dollar (US$) USG United States Government iii

MSME DEFINITIONS

Enterprise Category

Number of Employees

Annual Turnover, UAH

Annual Balance Sheet Assets, UAH

Micro <10 <1 million <500,000 Small <50 <15 million <5 million Medium <250 <100 million <15 million

Number of employees refers to the average number of workers during the reporting year.

MSMEs include producers (farmers). Producers should be classified as micro, small or medium-enterprises based on the number of FTE workers hired (permanent and/or seasonal) during the previous 12 months. If a producer does not hire any permanent or seasonal labor, s/he should be considered a micro-enterprise.

“Household farms” are small farms or large gardens, often directly around the house, run by a family and producing mostly for autoconsumption even though they may sell some of their produce. Ukrainian law defines them as noncommercial farms. AGRO generally considers them to be “microenterprises.”

“Family (peasant) farms” are a specific category of commercial farm defined in Ukrainian law that produce for market sale even though they may be family-owned and operated. They may be micro-, small or medium enterprises depending on how they fit the criteria above.

ACTIVITY DESCRIPTION

USAID’s Agriculture Growing Rural Opportunities (AGRO) Activity in Ukraine’s goal is to accelerate the economic development of rural Ukrainian communities with the greatest need through a better governed agricultural sector that encourages more productive, modern, and profitable micro, small, and medium agricultural enterprises (MSMEs) that are successfully integrated into competitive markets both in Ukraine and internationally.

The AGRO Activity comes at a critical time for Ukraine, when ensuring forward movement on enabling environment reforms and expanding agriculture sector gains are vital for economic growth and development. AGRO will harness and build on national and grassroots momentum in enabling environmental reform and scalable market system solutions primarily in the East and South to achieve accelerated economic development in rural communities with the greatest need.

High-Level Results

The high-level results for AGRO are:

• Increased revenue of agricultural small and medium enterprises (SMEs)

• Increased access for agricultural SMEs to financing

• Establishment of a fair and transparent agricultural land market (land reform)

• Reducing corruption in the agricultural sector

Development Hypothesis

The development hypothesis of the AGRO activity is:

IF AGRO encourages more productive, modern, and profitable small and medium agricultural businesses that are successfully integrated into competitive markets both in Ukraine and internationally through improved value chains and supporting functions AND improves the governance of the agricultural sector of Ukraine; THEN economic development of rural Ukrainian communities will be accelerated, supporting Ukraine’s Journey to Self-Reliance.

Activity Results Framework

This activity level results framework demonstrates AGRO’s links to the Mission’s results framework as well as the logical, causal relationships between the different levels of results.

COLLABORATING, LEARNING, AND ADAPTING PLAN

The AGRO Monitoring, Evaluation and Learning (MEL) system informs the Activity’s collaborating, learning, and adapting (CLA) approach to planning, implementation and evaluation. CLA allows for a high degree of flexibility to respond to changes in Ukraine’s operating environment, adapting lessons learned and ensuring successful Activity implementation. The CLA approach enables AGRO to understand how program implementation is affecting our partners and agricultural business environment and adjust accordingly.

As part of AGRO’s philosophy, CLA aims to:

• Improve the quality and relevance of all activities by grounding them in evidence;

• Make activities adaptive to new learning and changing contexts;

• Extend USAID influence through sharing knowledge and collaboration with development actors; and

• Catalyze learning among Ukrainian development actors to build capacity and facilitate country-led development.

AGRO prioritizes collaboration, learning, and adapting with individuals engaged by the Activity and those working within the Activity.