Attachment_J-13_-_Current_Contractor_Annual_Reports_from_2015_&_2016.pdf

November 20, 2017

This publication was produced for review by the United States Agency for International Development. It was prepared by The QED Group LLC.

USAID/UGANDA MONITORING,

EVALUATION AND LEARNING PROGRAM

“THE LEARNING CONTRACT”

YEAR FOUR ANNUAL REPORT

OCTOBER 1, 2016 – SEPTEMBER 30, 2017

1 | P a g e

THE LEARNING CONTRACT

Year 4 Annual Report (FY2017)

Contracted under AID–617-C-13-00007

USAID/UGANDA MONITORING, EVALUATION AND LEARNING PROGRAM

Prepared for Meaghan Wilson United States Agency for International Development/Uganda C/O American Embassy Plot 1577 Ggaba Road, P.O. Box 7007, Kampala, Uganda

DISCLAIMER

The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government.

CONTENTS

ACRONYMS

OVERVIEW AND SUMMARY

ABOUT THE LEARNING CONTRACT

EXAMPLES OF ACTIVITIES IMPLEMENTED BY THE LEARNING CONTRACT

YEAR 4 IN NUMBERS

OVERVIEW OF YEAR FOUR ACHIEVEMENTS

RESULT AREA 1 - USAID/UGANDA MEL SYSTEMS STRENGTHENED

Result Area 1 Main Accomplishments

RESULT AREA 2 - LEARNING AND ADAPTIVE MANAGEMENT PRACTICES

INSTITUTIONALIZED

Result Area 2 Main Accomplishments

RESULT AREA 3 - DEVELOPMENT NETWORKS STRENGTHENED

Result Area 3 Main Accomplishments

SUCCESSES, CHALLENGES AND APPLICATION OF LESSONS LEARNED

LOOKING FORWARD –YEAR 5 WORKPLAN

ANNEX 1: INDICATORS PERFORMANCE TRACKING TABLE

2 | P a g e

ACRONYMS

A/COR Alternate/Contract Officer’s Representative ADS Automated Directives Systems AMELPS Activity Monitoring, Evaluation and Learning Plans BOCY Better Outcomes for Children and Youth CDCS Country Development Cooperation Strategy CHC Communication for Healthy Communities CLA Collaboration, Learning and Adaptation DRG Democracy, Human Rights and Governance DO Development Objectives DQA Data Quality Assessment EG Economic Growth EYCD Education, Youth and Child Development FGD Focused Group Discussions FO Front Office FSN Foreign Service National FSO Foreign Service Officer FY Fiscal Year GDP Gross Domestic Product GOU Government of Uganda IP Implementing Partner IR Intermediate Result M&E Monitoring and Evaluation MEL Monitoring, Evaluation and Learning PAD Project Appraisal Document PMP Performance Management Plan PPD Policy & Program Development Office SAFE Supporting Access to Justice, Fostering Equity and Peace SCORE Sustainable Comprehensive Response for Vulnerable Children with their Families SIMS Site Improvement through Monitoring System SOCY Sustainable Outcomes for Children and Youth SOW Scope of Work OAA Office of Acquisition and Assistance OVC Orphans and Vulnerable Children RHITES E Regional Health Integration to Enhance Services in the Eastern Uganda RHITES EC Regional Health Integration to Enhance Services in Eastern and Central Uganda RHITES SW Regional Health Integration to Enhance Services in South West Uganda UEA Uganda Evaluation Association USAID U.S. Agency for International Development VSLA Village Savings and Loans Associations

3 | P a g e

OVERVIEW AND SUMMARY

ABOUT THE LEARNING CONTRACT

The purpose of the U.S. Agency for International Development (USAID)/Uganda Monitoring, Evaluation, and Learning Program (also known as “the Learning Contract”) is to provide monitoring, evaluation, and learning (MEL) support services and build the capacity of USAID/Uganda to implement USAID’s Collaborating, Learning, and Adapting (CLA) approach to strengthen the implementation of the Mission’s Country Development and Cooperation Strategy (CDCS). The GOAL of the program is to support USAID/Uganda in achieving its development results.

The CLA approach is a major component of USAID/Uganda’s strategies (CDCS 1.0 and 2.0). It enables a “living strategy,” providing guidance and reference points for implementation, learning, and course correction. The central function of CLA is to ensure that progress toward development objectives is guided by continuous learning and iterative adaptation of program and strategy implementation.

This report covers the 12-month period from October 1, 2016 through September 30, 2017 – Annual Report for project year four (FY2017). The Learning Contract is a five-year program under Contract No. AID-617-C-13-0007. It began in June 2013 under the QED Group, LLC.

The Learning Contract Result Areas

To establish the foundation of USAID/Uganda Mission as a CLA Mission, the Learning Contract addresses three result areas:

1. Strengthening MEL functions to support the Mission in making evidence-based decisions, improving organizational practices, and capacity for learning;

2. Supporting adaptation within the Mission’s staff and processes; and

3. Deepening coordination, collaboration, and synergy within the Mission and with other key stakeholders regarding program interventions, monitoring and evaluation, and lessons learned.

Implementation Approach

QED’s approach to implementing the Learning Contract is based on the following hypothesis: if focused, quality MEL products and services are provided; effective collaboration, learning and adaptation behaviors are institutionalized; and if strategic development partnerships are leveraged, then the Mission will achieve greater development results.

While we are ultimately concerned with USAID/Uganda’s ability to achieve its development objectives, we recognize that institutional adaptation can only happen by and within USAID and therefore is outside of the Learning Contract’s manageable interest.

Instead, we can have the greatest impact through Mission engagement that encourages informed decision-making, creating an environment and opportunity for USAID/Uganda to use knowledge and adapt responsively.

4 | P a g e

GOAL: Achievement of Development Results Enhanced

OBJECTIVE: Capacity of USAID/Uganda to Collaborate, Learn, and Adapt Strengthened

IR 1: USAID/Uganda MEL Systems Strengthened

IR 2: Learning and Adaptive Management Practices Institutionalized

IR 3: Strategic Development Partnerships Leveraged

Sub-IR 1.1: Quality Monitoring, Evaluation, and Learning Products Provided

Sub-IR 1.2: USAID/Uganda Staff Mel Skills Enhanced

Sub-IR 2.1: Learning and Adaptation Practices Built into Key Business Processes in the Program Cycle

Sub-IR 2.2: Models of Organizational Learning Cultures Amplified

Sub-IR 2.3: Adaptive Management Strategies Modeled

Sub-IR 3.1: Development Networks Strengthened

Sub-IR 3.2: Collaborative Monitoring, Evaluation, and Learning Initiatives Catalyzed

Sub-IR 1.3: Functionality of Information and Knowledge Management Systems Strengthened

5 | P a g e

EXAMPLES OF ACTIVITIES IMPLEMENTED BY THE LEARNING CONTRACT

6 | P a g e

7 | P a g e

YEAR 4 IN NUMBERS

Meetings hosted at the Learning Contract Office

Research briefers produced

Copies of 2016 USAID/Uganda Mission Annual Report printed

Copies of

USAID/Uganda Mission CDCS 2.0 printed

Drafts of

CDCS 2.0 PMP

Submitted

MEL trainings for IP staff delivered

Activity MEL plans developed and approved

Indicators assessed in formal DQA process

Quality MEL

Products submitted on time

Project

Appraisal Documents supported

Local

Consultants engaged

International Consultants engaged

Mission Retreat supported

IP Monthly

Communication Meetings supported

After-action

Reviews (AARs) facilitated

Mission staff exit interviews captured

Evaluations, assessments, studies and learning reviews

SIMS site visits supported

Chiefs of Party meetings supported

Development networks supported

8 | P a g e

OVERVIEW OF YEAR FOUR ACHIEVEMENTS

RESULT AREA 1 - USAID/UGANDA MEL SYSTEMS STRENGTHENED

To enhance monitoring, evaluation, and learning capacity for both the USAID/Uganda Mission and Implementing partners (IPs) staff, the Learning Contract continued to support initiatives leading to improved functionality of information and knowledge management systems, quality of performance data, and making knowledge and lessons learned more accessible across the Mission and partner communities.

Below is a summary of tasks accomplished under Result Area 1:

CDCS 2.0 final document produced and approved

2 drafts of CDCS 2.0 PMP developed and shared with USAID

Finalized SOWs for Private Health Sector and CHC evaluations, and phase 2 SOCY Baseline

Supported SIMS visits in the North & Southwestern OVC sites

Submitted 2nd draft of Nutrition Assessment

Submitted 3 deliverables from the Pardee Centre on sub-national data

Conducted Data Landscaping assessment

Supported development and modification of 10 Activity MEL Plans

Provided support for USAID/Uganda 2017

PPR

Completed Phase 1 SOCY baseline which was approved

Conducted desk review to inform Health Systems PAD

Operationalized the Learning Contract google site

Developed a concept note for Anthropological Study of Young Women in Uganda

Supported 6 Implementing Partners to conduct MEL capacity building training

Produced Public Version of the new strategy

(CDCS 2.0)

Provided Performance Reporting System support to implementing partners and the Mission

Completed fieldwork and drafted reports for 2017 Data Quality Assessments and completed 2016 DQA

Reviewed SOW for Mid-term evaluation of Rights and Rule of Law Activity (Freedom House)

9 | P a g e

Result Area 1 Main Accomplishments

1. Evaluations, Assessments, Learning Reviews and Special Studies

The table below provides a summary of evaluations, assessments, learning reviews, and special studies the Learning Contract either finalized or initiated in Project Year 4. Approved final products were uploaded to the DEC.

Task Comment

GAPP mid-term evaluation Final report approved December 19, 2016

Track TB assessment Final report approval October 27, 2016

Health Systems assessment Final report approved February 2017

Population Growth study Final report approved March 2017

Indoor Residual Spraying (IRS) evaluation Final report approved December 15, 2016

SRGBV learning process Final report approved November 2, 2016

CLA Perception and Progress analysis Finalized November 2016

EMOS learning review Approved December 2, 2016

Phase 1 SOCY baseline and QCA methodology

Approved June 2017

Nutrition Sector Assessment Final Report submitted August 2, 2017

Communication for Healthy Communities (CHC) evaluation

Scope of Work approved July 9, 2017

Private Health Sector (PHS) evaluation Scope of Work approved July 19, 2017

Phase 2 SOCY baseline Scope of Work approved July 7, 2017

2. Pardee Center Work on Sub-National GDP data

The Learning Contract, through the Pardee Center at the University of Denver, provided an initial survey of data resources in Uganda required for subnational future analysis (International Futures -IF-model). The Pardee Center noted that sub-national GDP data was not available and is a critical input for their IF model. In the absence of such data, the Pardee Center undertook an effort to estimate district-level GDP in Uganda. They developed spatial models and regressions to estimate district-level data using a variety of indicators of economic activity such as nighttime light intensity, population density, urban populations, education levels, and mobile phone use.

After evaluating the models that they developed, they determined that the Light Intensity Model produced the best estimates of district-level GDP in Uganda. These estimates have been approved by the Front Office and will form the basis of Pardee’s future modeling efforts.

10 | P a g e

The Pardee Center also gathered, cleaned, and uploaded 50 sub-national series into the model, which the Pardee Center considers to be very efficient given their timelines for sub-national work in other countries.

These estimates are expected to be widely used by USAID, the Learning Contract, the Government of Uganda, development partners, financial institutions, and others in future planning efforts.

Dispersion of GDP per capita estimates for select districts, thousands of US$ MER

A visual showing sub-national data gathering work completed by the Pardee Center

11 | P a g e

Participants’ Observations on activity MEL system

• Limited use of data collected for decision-making- need to adopt data use as a habit and do it routinely

• Data utilization happens at process level- use data in all aspects of the work

• Previously minimal intentional collaborations-this will improve work

• Need more pause and reflect meetings- to become more intentional in what we do

3. MEL Capacity Building in Qualitative Methods To build on the MEL training conducted in Project Year 3, the Learning Contract supported four Activities to conduct follow-on MEL training for their staff- RHITES EC, Rights and Rule of Law (RROL), Supporting Access to Justice, Fostering Equity and Peace (SAFE), and Better Outcomes for Children and Youth (BOCY). In total, 39 participants received training in qualitative methods, including Most Significant Change (MSC), Outcomes Mapping and Outcome Harvesting, and general MEL methodologies.

The MEL areas covered in the trainings include:

• MEL and program cycle;

• Operationalization of BOCY MEL system;

• MEL flagging system;

• Performance reporting;

• Data quality improvement;

• Qualitative methods;

• Developing a learning agenda and planning for learning moments; and

• Documenting lessons learned.

“This training has been very reflective. I looked back on assignments, engagements, and reports and tried to fit them into the MEL system and how these contribute to the overall progress and results achievements,” Participant /Trainee

“I will share this knowledge with the team members and document lessons learnt as part of implementation process itself rather than thinking about documentation of lessons as an afterthought,” Participant /Trainee

• Facilitating and documenting learnings and adaptions in a program is not a one-time task, it is also not a function of M&E staff only, but all staff and other actors should be involved throughout the Activity cycle;

• MEL training sessions provide a platform for active engagement and stimulation of staff to question the AMELP status quo, and an avenue for generating new ideas and learning questions to inform decision-making and adaptation of new ideas and knowledge;

• It is anticipated that the knowledge gained by the implementing team during MEL training will be used in their daily Activity work to support other staff in CLA, MEL, and documentation, and

Lessons Learned

12 | P a g e improve Activity results, and skills in qualitative methods that will enable them to track and measure changes in behaviors, practices, and attitudes.

4. CDCS 2.0 Performance Management Plans (PMP) Development Following the approval of the new CDCS in December 2016, and the ADS guidance which requires the agency to construct a new PMP six months from strategy approval date, the Learning Contract staff continued to work with different Mission technical working groups to develop the PMP for the new strategy. The Learning Contract staff participated and led the design of different sections of the PMP, including the M&E architectural design, identification of the different indicators, and writing the learning questions. To bolster the Mission’s PMP development process, the Learning Contract brought onboard a PMP collaborator who worked directly with Mission MEL teams and the Learning Contract teams to map and coordinate all processes leading to the finalization of a new PMP.

Following the Mission’s feedback on the first draft submitted in the 3rd quarter, the Learning Contract thoroughly revised and reorganized the PMP, further developing measurement methods, indicators, and ensuring overall coherence. The Learning Contract submitted a 2nd draft PMP on August 25, which included a revised MEL architecture, proposed indicators for guiding principles, refined proposals for systems measurement, and more direct language for adaptive performance management.

• Because of the Learning Contract’s iterative PMP development, the Mission is now better poised to implement its Country Development Cooperation Strategy, and better poised to manage its success while adapting to learning during implementation;

• The Learning Contract has also learned that performance management planning requires practical ideas tied to strategy. With the new CDCS, the PMP must present innovative methods for learning about and managing the implementation of an integrated strategy;

• Through developing the PMP, the Learning Contract has strengthened current and future MEL systems within USAID/Uganda;

• Through consultations with the Mission, the Learning Contract has learned and adapted to focus on how the PMP can be used as a tool to ensure informed decision making.

The Learning Contract will continue to look at the practicalities of how USAID/Uganda and its partners implement and continue to consider how we use learning to improve performance.

13 | P a g e

5. Data Quality Assessments (DQA)

The purpose of DQA is to establish the strengths and weaknesses of data, as determined by applying the five data quality standards of Validity, Integrity, Precision, Reliability, and Timeliness, and the extent to which the data is trusted to influence management decisions.

In Project Year 4, the Learning Contract concluded the 2016 Data Quality Assessment (DQA) and conducted the 2017 DQA for the Family Health and Economic Growth technical offices. A total of 32 Family Health indicators, covering 10 Activities were assessed, as well as 9 Economic Growth indicators, covering 7 Activities.

The 2017 DQA process focused on data reported during the fiscal year (FY) 2016 and data for the newly activated Program Performance report (PPR) indicators to be reported during FY 2017.

The majority (66%) of Family Health indicators were rated Acceptable if corrections are made, implying there is room for improvement. Only 34% of the indicators assessed were ‘Acceptable’. Most data quality (DQ) issues identified were under Reliability (16 indicators) followed by Validity (12) and Precision (7) as shown in the graphic below.

For Economic Growth Indicators, overall data for 3 out of 9 indicators assessed (33%) were rated ‘Acceptable’. The remaining 6 indicators (67%) had data that was ‘Acceptable if corrections are made’.

No indicator data was rated ‘Not acceptable’; as presented in the visual below.

14 | P a g e

• Aggregating indicator data from IPs with varying quality will often bring down the overall rating especially if one on the IPs obtains a ‘Not Acceptable’ score;

• IPs that have established (and strictly adhere to) internal data verification and certification protocol consistently have better quality data;

• IPs that have distributed standardized data collection tools (printed forms) to field-based partners, generally have better quality data in terms of completeness and consistency.

6. Site Improvement through Monitoring System (SIMS) Activities

The Learning Contract supported USAID/Uganda EYCD technical office to conduct SIMS OVC visits for Uganda Private Health Support Program (UPHS), Sustainable Outcome for Children and youth (SOCY), Better Outcomes for Children and Youth (BOCY) and Sustainable Comprehensive Responses (SCORE) Activities in the North, Southwest and Central regions. Also, the Learning Contract supported the Family Health team to conduct SIMS visits in sites supported by Regional Health Integration to Enhance Services in South West (RHITES SW).

During SIMS visits, a participatory approach is adapted to engage and generate responses from program staff on-site using the standard SIMS tools. The approach allows the organization and staff to self–assess, learn, and challenge themselves to improve planning, documentation, and case management records and data quality management to ensure quality and sustainable services are provided to their beneficiaries.

15 | P a g e

Overall, personnel at assessment sites had a good understanding of their program interventions, including approaches of OVC, HIV/AIDS and Family Health service delivery. Various policy guidelines, including standard operating procures (SOPs,) exist in most of the sites visited.

• Some lead IPs don’t appreciate holistic/integrated programming for targeted children – highlighting no funds for activities such as malnutrition screening and family planning. The IPs may need to exploit existing opportunities like public health facilities to collaborate and strengthen referral related activities between partners;

• For the SIMS team, failure to reflect on the previous site performance in comparison to the current assessment makes the team look unfocused- some background knowledge on previous assessment should be used by the team to guide discussion and better mentor site staff;

• Case management records are inconsistent in most sites – there is a need to strengthen records keeping and documentation with required information especially about households in the programs;

• Staff working with AVSI under the SCORE activity replicated activity processes, practices and activities at the household and individual levels, and these include VSLA, parenting skills, couple’s communication, and nutrition management;

• The implementation of the integrated health activities is very complex and requires a lot of staff time to ensure that all components of the health system are adhered to;

• While they are complex to implement, they also require a lot of innovative ideas. It is important that the activities factor in a time for reflection and learning to minimize staff burn out.

District Probation Office- Gulu for BOCY Activity SIMS Visit

16 | P a g e

RESULT AREA 2 - LEARNING AND ADAPTIVE MANAGEMENT PRACTICES

INSTITUTIONALIZED

This result area is geared towards creating a conducive environment for learning and adaptation by providing examples, modeling behaviors, and supporting the use of appropriate incentives and motivations for change management.

To ensure that learning and adaptation are ingrained in the Mission’s work, the Learning Contract accomplished the following tasks, as summarized in the table below.

Supported October 2016 Mission Retreat and initiated preparations for November 2017 Mission Retreat

Supported drafting of 5 PADs (Demographic Drivers, Resilience, Health Systems, DRG, and Market Systems)

Developed USAID/Uganda Mission Transition Management Action Plan for the new strategy

Produced 2016 USAID Mission Report to the Ugandan People

Developed Systems Thinking Course for Mission and IP Staff

Completed and submitted 10 Research briefs on Northern Uganda

Supported EG Office to documented The Village Documentary

Supported monthly Chief of Party meetings

Supported RECO/PIN close-out event

Initiated preparations for the State of Ugandan Agriculture Forum scheduled for September 2018

Conducted 3 Mission Staff exit interviews

Supported field Learning visit for enhancing Climate Resilience of Agricultural Livelihood Project Change

Conducted Economic Growth Research Agenda review

Produced CDCS 2.0 Journey Briefer

Conducted Resilience Project Literature Review and Knowledge Management

Developed Presentation on CDCS 2.0 for new Mission staff

Initiated Portfolio Reviews for 2017

Facilitated Better Outcomes Learning Agenda workshop

Result Area 2 Main Accomplishments

1. CDCS 2.0 Final Copy Production Following approval of the new strategy (CDCS 2.0) in December 2016, the Learning Contract continued to engage the Mission on how to operationalize and publicize the new strategy. To support the Mission’s efforts to widely share its vision for USAID assistance in Uganda’s development and use the strategy as a tool to discuss partnership opportunities, the Learning Contract conducted a final review of the strategy, including copy editing and layout changes for the public version of the strategy. After undergoing a competitive process to vet vendors, 1000 copies of the strategy were printed and delivered to the Mission for distribution.

The Learning Contract also composed a brief regarding the CDCS development process, outlining critical junctures, significant roles, and other highlights. As a result, the Mission can now share their

17 | P a g e learning process with stakeholders and spark conversations around the lessons they learned in the development of the new strategy.

Public version of CDCS 2.0 Book

2. Project Appraisal Documents (PADs) Development Support

Immediately after CDCS 2.0 was approved, the Learning Contract worked closely with the Mission to develop five Project Appraisal Documents (PADs) – Demographic Drivers, Resilience, Health Systems, DRG, and Market Systems.

The Learning Contract support included:

Research and analysis;

Measurement, conceptualization, and indicator selection; and

Development and prioritization of a list of future research and learning questions.

In Project Year 5, The Learning Contract will continue to support finalization of PAD development through refinement of theories of change, concluding MEL plans and frameworks, facilitating discussions on project geographic focus, aligning Mission performance management to projects, and providing contextual problem analysis research. The Learning Contract will also support the Program Office directly in its organization of PAD authorization and attribution across the CDCS.

18 | P a g e

3. Staff Transition (Staff Exit Interviews) During the year, the Learning Contract conducted 3 staff exit interviews. The exit interviews included conversations with USAID/Uganda’s departing Private Sector unit leader, outgoing Chief of Party of Political Competition and Consensus Building, as well as the Learning Contract organizational learning advisor.

The purpose of conducting the exit interviews is part of the CLA approach to help us learn from departing staff about some of the challenges, successes, and recommendations they might have. We anticipate that the Mission will use these exit interviews as part of their new staff orientation and for future reference.

Below are reflections on USAID/Uganda achievements on CLA, how the Mission has shifted its thinking, and its organizational needs heading into the new strategy.

“Moving from CDCS1.0 to CDCS 2.0 has seen a shift in the way the Mission does business, we are beginning to focus on people, slowly identifying the local champions, working on local solutions with local companies and to identify what are their issues and how do the communities take ownership of these activities.” Private Sector unit leader

“We have seen the role of the Learning Contract evolve from basic tasks to more integrated tasks, such a;

creating learning moments, asking the right questions for evaluations, learning reviews, translating learning into business processes and getting better and beyond learning.” KMOL Advisor –Learning Contract

“Organizational learning for the Mission has meant reminding ourselves of what we have learned, take intentional pause and reflect moments to understand what evidence do we have, what adjustments do we want to make going forward, it also means getting beyond monitoring and evaluation and reporting and thinking about building enough evidence, what trends are we seeing and getting a hand on who else is asking these questions.”

KMOL Advisor –Learning Contract

“If we are to make development changes in Uganda, we are going to adopt learning as a practice, apply leadership, organizational development, and change management principles to the development work that we do.” KMOL Advisor -Learning Contract

Departing Chief of Party reflections on USAID/Uganda PCCB activity

“To advance integrated programming, USAID/Uganda should be less prescriptive on implementation but tolerate flexibility and unexpected adjustments to implementation.”

“Continuous communication between implementing partners and the Mission are key to mutual learning and good program management.”

19 | P a g e

4. Village Agent Model Documentary

In May 2017, the USAID/Uganda Economic Growth team asked the Learning Contract to solicit consultancy services of a firm to produce a video documentary to highlight the Feed the Future Uganda Commodity Production and Marketing Activity’s Village Agent Model. USAID/Uganda uses the Village Agent Model to improve farmer access to markets and needed services and to reduce commodity supply chain interruptions.

The Learning Contract carried out the procurement process in June 2017 and selected a firm that presented best value for money to produce this video. The video documentary is aimed at telling the story of 2 different farmers who have not used an agent before, or were reluctant to trust an agent, to show the benefits of this model. The editing of the video was completed in September 2017, submitted to the Economic Growth team, and the procurement process was closed.

Moving forward, the Learning Contract will continue to support such innovations which provide the Mission a platform to demonstrate some of its successful models.

5. CDCS 2.0 Transition Management

In June 2017, the Learning Contract worked with PPD to arrange 4 focus group discussions (FGDs) to gain a better understanding of USAID/Uganda's staff perspectives of transitioning to the new CDCS.

Key aspects of the FGDs are listed in the table below:

Focus Groups Positions represented AOR/CORs, MEL Group, Office Directors and PAD

Coordinators Offices represented EG, Health, DRG, PPD, EYCD, and OAA Total number of participants 21 FSO/FSN representation FSO: 8; FSN: 9, and Others: 4

The analysis found that 100% of FGD participants were excited and invested in the field-focused, integrated model of development laid out in the CDCS. The analysis also identified key challenges and bottlenecks. Everyone from Office Directors, AORs, FSNs, and FSOs, had comments clustered around a similar theme: the need for a clear “road map” on processes, roles and responsibilities, and lines of accountability. The report also identified potential quick wins that could address some of the inefficiencies discussed and improve organizational processes.

The Learning Contract then prepared the full notes as well as an executive summary, analysis and suggestion document for PPD and FO. In addition, all results were presented and discussed via PowerPoint presentation with FO.

Subsequently, the Learning Contract prepared a Transition Management Action Plan SOW for a small team of contractors to conduct an assessment and prepare a Transition Management Road Map with

20 | P a g e milestones to guide the Mission toward the procedural and organizational modifications required to achieve the multidimensional objectives described in the new CDCS, and increase the efficiency of USAID business processes and practices. Two candidates were identified towards the end of September and are anticipated to start in November 2017.

6. Learning Contract Research Efforts

The Learning Contract is supporting regional development research efforts through background evidence generation (research briefs) and investigations into more specific topics. The Research and Learning team completed seven (7) topical briefers for the Northern Uganda regional development efforts.

These include briefers on:

1) Electrification;

2) Oil & Minerals;

3) Land Tenure System;

4) Trade Linkages;

5) GOU Investments in Northern Uganda;

6) Soils of Northern Uganda; and

7) Watershed Management in Northern Uganda.

The Research team also produced a revised briefer for the Municipalities which expanded upon information included in the previous regional briefers.

These topical briefers were produced to help fill in identified knowledge gaps and better inform the contextual evidence gathered to prepare an investment plan for the region. Some of these topical briefers have been shared with the relevant teams at USAID.

The major lesson learned is that there is not a lot of publicly available information about some of these topics and up-to-date statistics are difficult to find. Some offices or individuals within USAID have better information, but we need to be strategic in finding out where this information is held.

In the future, we intend to clarify the use of these products from the beginning. These were originally only thought to be used as information for the Mission Director, but after they were created there was an idea to share them more widely. We should be more deliberate in making these products for the specific audience, when possible.

21 | P a g e

RESULT AREA 3 - DEVELOPMENT NETWORKS STRENGTHENED

The Learning Contract worked to strengthen and develop stakeholder networks and communities of practice across thematic or sector-based platforms, through application of innovative tools and approaches to promote stakeholder engagement across the program cycle, and develop a better understanding of good practice or unique MEL capabilities. The Learning Contract assisted USAID/Uganda in co-creating development knowledge, leveraging development partner capacities, influencing policy and implementation approaches while improving and promoting beneficial partnering practices.

Below is a summary of tasks accomplished under result area 3:

Hosted 195 meetings at the Learning Contract’s office space

Provided technical support to March 2017 African Evaluation Association (AFREA) Conference

Supported Agricultural Market Systems event

Provided technical support to 6 Northern Uganda Regional Development meetings

Organized GAPP midterm evaluation stakeholders’ validation meeting

Supported design and facilitation of the MERLIN workshop

Supported Understanding the Influence of Social Context on Adolescent Trajectories event

Supported Uganda Agricultural Market Systems event

Supported Private Sector Development Partners Group meeting

Supported USAID VMMC progress review and ENR & Climate Change monthly meetings

Provided technical support to women and youth workshops in Northern Uganda

Hired firm to support Research work in Northern Uganda

Supported quarterly USAID HIV Implementing Partners' meeting

Monthly Environment and Natural Resources Sector Development Partners Group (ENRDPG) meetings

Facilitated Final PCCB Advisory Board Meeting (Reflection on Performance of

PCCB)

Produced 1000 copies of the Public Version of CDCS 2.0

Printed 2000 copies of 2016 USAID Uganda Mission Report to the Ugandan People

Printed 1000 copies of USAID/Uganda folders

Support Mission’s monthly meetings

Supported EYCD office IPs experience sharing meeting (OVC and DREAMS Activities)

Supported Resilience Support Unit and Refugee Donor Group

22 | P a g e

Result Area 3 Main Accomplishments

Some samples of strategic development partnerships initiated through the Learning Contract in Year 4 are discussed in details below.

1. USAID Mission Uganda Report to the People of Uganda

The Learning Contract supported the Mission to produce their first-ever Report to the Ugandan People – a summary of some of the most important U.S. assistance programs in Uganda in 2016. The stories contained in the report highlight the many successes of Ugandans across the country and demonstrate the strength and value of the relationship between the United States and Uganda.

Collaborative, a communications firm based in the USA, was sub-contracted by the Learning Contract to offer writing, editing, and graphic design services towards the production of the report. Printing of 2000 copies of the report was also done by the same firm, though it was originally planned to be printed locally in Uganda.

The report is approximately 40-50 pages of text with heavy imagery, easily readable and visually appealing, with photography and graphics illustrating the Mission’s work.

To achieve this, the Learning Contract contracted two photographers from February-March 2017 to produce high quality, professional images that highlight USAID/Uganda’s programs, for purposes of upgrading public awareness and outreach efforts, as well as communicating USAID’s work and impact to the public. The Learning Contract coordinated and facilitated the photography processes throughout the country. The photographers submitted over 4000 images, some of which were used in the Report to the People.

In addition, 1000 USAID/Uganda folders were printed locally and delivered to the Mission in September 2017.

2. Regional Development Meetings to Push Mission’s Regionalization Strategy

The first regional development meeting took place in Arua, Northern Uganda on March 23, 2017. The Northern Uganda regional development meetings are part of the Mission’s stakeholder consultations efforts across several geographic regions to enable consultative development of USAID projects and

23 | P a g e activities in Uganda. The Learning Contract organizes these regional stakeholder consultations in collaboration with USAID.

The meetings brought USAID together with key stakeholders in the North to develop a regional Steering Committee that will guide future development priorities in the region. Members of local government, civil society, opinion and religious leaders as well as the Prime Minister’s Delivery Unit participated.

Additional sub-regional consultation meetings in West Nile, Acholi, and Lango were held to identify sub-regional priorities, and Women and Youth workshops in Gulu were facilitated to capture priority areas from these groups and to ensure that the regional development process is more inclusive.

The Learning Contract support to the regional development meetings included:

• Undertaking research tasks on additional themes;

• Updating briefers (information sheets) on sub-regions of Northern Uganda;

• Making presentations and facilitating Steering Committee or stakeholder meetings;

• Recruitment and supervision of research fellows.

• A call for more district-disaggregated data e.g. soil maps, agro-processing / investments in value addition, etc;

• There is need for leadership capacity development and improvements in governance, transparency, and accountability;

• There is need for research and analysis of push and pull factors that will transform Northern Uganda;

• Strategic planning is needed to ensure an inclusive, well thought out process.

Participants during Regional Development Meeting sessions in Arua district

24 | P a g e

3. AFrEA 8th International Conference

During the 2nd quarter of Year 4, the Learning Contract played a lead role in the coordination and management of the 8th AfrEA International Conference. QED HQ personnel and Learning Contract staff chaired a key topic “Leveraging Capacity Building Knowledge” and several other sub-topics and sessions

- highlighting stakeholder engagement in evaluation SOW development, validating findings and drawing conclusions as part of USAID’s approach to move the locus of learning. QED’s additional involvement included participation in pre-conference training, documenting take-aways/rapporteuring, and highlighting the Learning Contract’s knowledge products.

The visibility of USAID/Uganda and QED in evaluation practice and knowledge management was positive. Also evident was the partnership between Uganda Evaluation Association (UEA) and QED.

4. The State of Ugandan Agriculture Event/Forum

The State of Ugandan Agriculture event is scheduled for September 2018 so work on this event is on-going. Activities during the 4th quarter included facilitating a competitive hiring process for a high level agriculture consultant and a project manager for the forum, preparation of draft work plans and event budgets, hosting meetings with USAID staff, IPs, and other partners to solicit and incorporate their feedback into the planning of the event, revisions to the event concept note, development of a business survey for USAID review and revisions, scoping initial research for background information to the event, and preparation of the event's organizing committee within the Ministry of Agriculture, Animal Industry and Fisheries.

This event builds upon lessons learned from the State of the Ugandan Child events previously supported, but the needs and audience for this event are considerably different than the Child Forum so it has been an iterative process to determine the best fit for the event. The Learning Contract continues to adapt with USAID as the vision for the event takes shape.

5. Collaboration Meeting Space at the Learning Contract

The new Learning Contract office space is near the Mission and has space for the Mission staff to work from, a conference room with a capacity of 70 people, a ‘thinking room’ for either individuals or teams to brainstorm and be creative, and two small meeting rooms.

The Learning Contract supported and hosted about 200 different meetings as part of USAID/Uganda’s collaborative work, both within the Mission and externally with its IPs and other key stakeholders. The collaboration space has attracted a range of users, including Mission staff, IPs, Private Sector actors, Government of Uganda, and other stakeholders with a broad spectrum of purposes.

25 | P a g e

26 | P a g e

SUCCESSES, CHALLENGES AND APPLICATION OF LESSONS

LEARNED

Application of Lessons Learned

Being strategic and focused on improvements (re: planning with COR)

Being open to failure and risk (trust staff with increasingly important tasks)

Optimize the new office space to enhance Result Area 3

Provide clarity and use assessments, tools, etc (to clarify roles, scope, etc)

Consider how to better inform the team on conclusions and outcomes. Focus on content of

“What are we learning?” Become each other’s ambassadors. Embrace attitude of “all of us” and draw on individual strengths.

Talk to COR early on to not get overloaded. Help make the decision (“add to the wall” as a

Positives

Good working relationship with the Mission and IPs

Positive feedback received (helpful and useful)

Embrace of multiple methods by stakeholders

Collective participation/shared responsibility

Broad perspective/ input with tools that led to a changing perspective

Responsiveness and flexibility

New office structure; working together

New COR(s) (action: be strategic and focused on improvements)

New staff

Trusted with more important tasks (action: be open to failure and risk)

Team fusion of Learning + M&E

Better communication

Challenges

Not managing expectations well

Unclear roles, scope, decision points, etc.

Last minute requests or changes

Too many decision-makers

Lack of information sharing

Lack of communication and transparency

Need more effective processes

Focus on fixing problems

Difficult to monitoring results of Component 3

27 | P a g e new workplan task/ future accomplishment). Conduct quick touch-bases to orchestrate and organize.

Introduce team to others in the Mission/share relationships

Consider what partners get out of Component 3. Re-explore interactions with GOU and UEA

(i.e. technical working groups). Track and follow up on individual, informal outreach. Conduct

Network Analyses. Engage with universities.

Report back on meetings and interdepartmental sharing

Take online courses; identify and share these

Ruth A, Jordan, and Jino to form a Team Spirit task force

Determine the mechanism to track result 3

28 | P a g e

LOOKING FORWARD –YEAR 5 WORKPLAN

Result Area 1:

USAID/Uganda Monitoring, Evaluation, and Learning Systems Strengthened

Result Area 2:

Learning and Adaptation Practices Institutionalized

Result Area 3:

Strategic Development Partnerships Leveraged

Summary of Year Five Tasks

• Conduct 3 evaluations (CHC, PHS and SAFE Evaluations)

• Phase 2 SOCY Baseline

• 2017 Data Quality Assessments

• Site Improvement Monitoring System (SIMS) Support

• CDCS 2016-2021

Performance Management Plan (PMP)

• Activity Monitoring, Evaluation, and Learning (MEL) Plans

• International Futures (IF) Modeling

• MEL Architecture for the New PMP

• MEL capacity building training

• Performance Reporting Systems support

• Learning Contract Google Site maintenance

• Data Landscape assessment (M&E architecture assessment)

• Project Appraisal Document (PAD) development

• Assistance in applying CLA in key business processes

• IP CLA support

• Staff Transition support

• Strategic advice and tools

• Mission Retreat and all-hands learning events

• Mission Portfolio Reviews

• Transition Management Action Plan

• Mission baselining and target setting for the new PMP

• Research Tasks in Quarter 4

• Finalize the Village Agent Documentary

• Regional Development Stakeholder engagements

• Research for Northern Uganda Regional Forum

• National State of Uganda Agriculture Forum

• Sector Working Groups and Priority Development Networks

• Wider Mission collaboration and coordination efforts

• Monthly Chiefs of Party meetings

• Topical/sector review sessions

• On-demand organization of workshops, conferences and consultations

• Inclusive post evaluation/assessment/resear ch, stakeholder validation, knowledge sharing, and feedback events

• Tactical learning and KM support to Agency-supported learning events and/or conferences

• USAID DOC office monthly meetings

29 | P a g e

ANNEX 1: INDICATORS PERFORMANCE TRACKING TABLE

USAID/Uganda Monitoring, Evaluation and Learning Program Indicator Performance Tracking Table

Performance Indicator Baseline Baseline 2015 2016 2017 2018

Comments Year Value Target Actual Target Actual Target Actual Target Actual

IR 1.0: USAID/Uganda MEL Systems Strengthened

1.0.1: Percent of USAID/Uganda Mission staff and IPs that have confidence in the data available to make design and adaptation decisions.

2014 58% NA 65% 31% NA 50%

2016 MELA: Qn 11 for mission and Qn 11 for IPs.

Baseline is from the MELA conducted at launch of the program.

1.0.2: Percent of 24 learning agenda questions listed in the CDCS that have been explored with Learning Contract assistance.

2014 0 30% 33% 55% 59% 55% 0 60%

Mission also uses other contractors to conduct evaluations, studies and assessments to answer learning agenda questions.

From an existing list of 24 learning agenda questions contained in CDCS 1.0.

Sub-IR 1.1: Quality Monitoring, Evaluation, and Learning Products Provided

1.1.1 Percent of USAID/Uganda

staff that are satisfied with the quality of products and services provided by the Learning Contract

2014 0 NA 40% 32% NA 55%

2016 MELA: Qn 15 for mission and Qn 19 for IPs.

Baseline is from the MELA conducted at launch of the program.

1.1.2: Number of quality MEL products delivered on time. 2014 15 20 23 20 19 20 41 20

These include evaluations, studies, assessments, learning reviews, ONA, guides, briefers, case lets and PPR.

Sub-IR 1.2: USAID/Uganda Staff MEL Skills Enhanced

1.2.1 Percent of USAID/Uganda

Mission staff and IPs who report an improvement in their abilities to perform MEL functions in the last 12 months.

2014 0 NA 40% 45% NA

50%

2016 MELA: Qn 16 for mission and IPs 20. Baseline is from the MELA conducted at launch of the program.

30 | P a g e

Indicator Performance Tracking Table

Performance Indicator Baseline Baseline 2015 2016 2017 2018

Comments Year Value Target Actual Target Actual Target Actual Target Actual

1.2.2: Number of quality activity MEL plans developed by USAID/Uganda with the support of the Learning Contract.

2014 20 20 31 10 14 10 10 6 Reported annually but tracked quarterly as part of progress reports.

Sub-IR 1.3: Functionality of Information and Knowledge Management Systems Strengthened

1.3.1 Number of USAID/Uganda

staff using information and knowledge management systems.

2014 32 60 65 60 67 60 71 60

These includes Mission staff accessing the PRS for various purposes, Chiefs of Party who verify data, IP M&E staff who manage Activity data.

Reported quarterly as part of progress reports. Baseline year is 2014

1.3.2: Percent of USAID/Uganda Mission staff and IPs satisfied with information and knowledge management systems.

2014 0 NA 40% 38% NA 50%

2016 MELA: Qn 6 for mission and IPs 8. Baseline is from the MELA conducted at launch of the program.

IR 2.0: Learning and Adaptive Management Practices Institutionalized

1: Percent of USAID/Uganda Mission staff and IPs who perceive advancements in CLA and/or institutional changes.

2014 0 NA 38% 64% NA 88%

2016 MELA: Qn 13 for mission and IPs 14. Baseline is from the MELA conducted at launch of the program.

2: Percent of USAID/Uganda Mission and IP staff reporting that activities have been designed or adjusted as a result of CLA.

2014 0 NA 25% 76% NA 85%

2016 MELA: Qn 9 for mission and Qn 12 for IPs. Baseline is from the MELA conducted at launch of the program.

Sub-IR 2.1: Models of Organizational Learning Created and Shared

2.1.1: Number of model organizational learning examples documented (success stories, case studies, white papers, etc.).

2014 0 5 1 4 1 4 2 2 Model learning cultures documented by the Learning Contract – Case lets.

IR 3.0: Strategic Development Partnerships Leveraged

31 | P a g e

Indicator Performance Tracking Table

Performance Indicator Baseline Baseline 2015 2016 2017 2018

Comments Year Value Target Actual Target Actual Target Actual Target Actual

3.0.1: Number of USAID collaborative meetings and events hosted or facilitated by the Learning Contract

2016 0 NA 100 195 80

New indicator. Reported Annually but data tracked quarterly as part of progress reports. Baseline is 2016.

Sub-IR 3.1: Development Networks Activated and Strengthened

3.1.1: Number of development networks (working groups) supported by Learning Contract

2014 0 5 5 4 5 4 5 5

Sectors include Health, Education, Agriculture, Private sector, Energy, Environment etc.

3.1.2: Percent of USAID/Uganda Mission staff and IPs who perceive participation in development networks as beneficial to development work

2014 0 NA 50% 20% NA 65%

2016 MELA: Qn 15 for mission and IPs 6. Baseline is from the MELA conducted at launch of the program.

32 | P a g e

U.S. Agency for International Development Plot 1577 Ggaba Road, P.O. Box 7007, Kampala, Uganda www.usaid.gov

Revised April 7, 2017

This publication was produced for review by the United States Agency for International Development. It was prepared by The QED Group, LLC.

USAID/UGANDA MONITORING,

EVALUATION AND LEARNING PROGRAM

“THE LEARNING CONTRACT”

YEAR THREE ANNUAL REPORT

OCTOBER 1, 2015 – SEPTEMBER 30, 2016

THE LEARNING CONTRACT

YEAR THREE ANNUAL REPORT

Contract Number: AID-617-C-13-00007

Submitted to: Phillip Greene United States Agency for International Development/Uganda C/O American Embassy Plot 1577 Ggaba Road

P.O. Box 7007, Kampala, Uganda

Submitted by: The QED Group, LLC 1820 N. Fort Myer Drive Suite 700 Arlington, VA 22209

USA

DISCLAIMER:

The authors’ views expressed in this document do not necessarily reflect the views of the United States Agency for International Development or the United States Government.

USAID/Uganda The Learning Contract Year Three Annual Report

ACRONYMS

AAR…