APPENDIX 2 - Results Monitoring Plan Template.docx

Appendix 3: Results Monitoring Plan (RMP) Template for Application

Instructions: Please refer to Attachment 1: Results Monitoring Plan Guidance to ensure you include the correct information in each section.

I. Project Overview

Project Title:

Organization:

a. Problem or Opportunity Statement and Policy Relevance:

b. Project Description:

c. Project Goal and/or NEA Strategic Framework

d. Project Objectives

II. Theory of Change and Logic Model Please refer to the suggested template in Attachment 2.

III. Project Workplan

Activity # (Align to Logic Model)

What will this Activity/ Milestone Achieve? When will it start?

(Outputs and short-term outcomes in your logic model) Location Where will this take place?

(Region/ province/ virtual or in-person) Public Event?

(Yes/No) Activity/ Milestone Completion Date (When will this be completed?

MM/YYYY)

☐Virtual ☐In-person – location:

☐ Yes ☐ No

☐Virtual

☐ Yes

☐Virtual

☐ Yes

☐Virtual

☐ Yes

IV. Results Monitoring Plan (RMP) - Project Indicators Please refer to Attachment 1 for guidance on how to fill out each field.

Project Objective 1:

Definition:

Disaggregation:

Target:

Baseline:

Target per Fiscal year:

Target Rationale:

Data Source:

Activities:

Project Objective 2:

Definition:

Disaggregation:

Target:

Baseline:

Target per Fiscal year:

Target Rationale:

Data Source:

Activities:

V. Planned Evaluation(s) [if applicable]

VI. Attachment(s)

1. Results Monitoring Plan Guidance

2. Theory of Change and Logic Model Template

3. Selecting High Quality Performance Indicators

4. Most Significant Change Indicator Guidance

Attachment 1 – Results Monitoring Plan Guidance

This attachment provides instructions and examples of how to fill out each required section of the scope of work template. Example entries are in blue.

Problem or Opportunity Statement and Policy Relevance: Provide the problem or opportunity to be addressed. Available in the announcement and/or the invitation to apply.

Project Description: Provide brief (1-5 sentences) project description. How are the project activities related to a foreign/national security goal? What will your project use the funding for? What result(s) do you intend to achieve through this project? How will you know whether you have achieved the desired result(s)? And how will you show success?

Project Goal and Objectives: Insert the project’s connection to higher level strategic goal, policy aim, or intermediate results as provided in the announcement or invitation to apply as the Project Goal. Insert the project’s Specific, Measurable, Achievable, Realistic, and Timebound (SMART) Objective(s).

Theory of Change and Logic Model: Insert the project’s Theory of Change and Logic Model in this section or at the end of the RMP document, include assumptions and external factors.

Project Workplan: The project will carry out the main activities referenced in your completed Logic Model (mentioned above) to achieve the Project Objectives. These Activities/Milestones will become the RPM Workplan in MyGrants. They will become a contractual obligation to either complete or modify in coordination with NEA during the project.

Please complete the table outlining all planned project activities with a corresponding timeline under each project objective. Please identify target completion dates (milestones) for each activity. We understand that these dates are estimates and may change.

Milestones are the specific events/activities/tasks that need to take place during the period of performance to achieve the Project Objectives; if these milestones are not completed, outputs and outcomes will not occur. Milestones are binary in that you either complete them or you do not. The corresponding timeline and associated deliverables will be used to monitor progress.

Results Monitoring Plan (RMP):

Project Indicators should monitor the progress occurring over time towards Outputs, Outcomes, and Objectives from your logic model. This RMP will become a contractual obligation to work towards or modify in coordination with NEA during the project.

Progress towards both project Work Plan and RMP must be reported on in accordance with your project reporting schedule, and more details for suggested narrative reporting will be provided during the kick-off of the project.

Proposed indicators should be direct, objective, adequate, and practical measures of the result(s) to be achieved. See Attachment 3 - Selecting High Quality Performance Indicators for more information.

Please do not exceed 20 total indicators, and ensure each indicator is relevant to the project and useful for adaptive management.

For RMP indicators, please include the following:

· 1-6 Total Indicators per Objective (see Program Indicator List Annex from the announcement). These should include:

· At least 1 F Standard Indicator per project; a suggested sub-set most relevant to the project area is included in the announcement or an Annex . The full list of available Foreign Assistance Standard Indicators is available here: Foreign Assistance Resource Library - United States Department of State

· At least 1 Outcome indicator per Objective (the Most Significant Change (MSC) can count as an outcome indicator). See Attachment 4 – Most Significant Change Indicator Guidance for more information.

· Outcome indicators monitor longer term project results as articulated in the Logic Model; outcomes are the changes made by the outputs of the project and help NEA convey project effects to internal and external stakeholders.

· 1 - 4 Output indicators per Objective

· Output indicators track immediate or near-term results or deliveries of products, services, or trainings. They should not include basic activities like holding meetings which can be tracked via the Project Workplan.

· When applicable, 1 MSC Indicator per project (see Attachment 4 - Most Significant Change Indicator Guidance below).

· If necessary, suggested project custom indicators along with a rationale for their inclusion

Please note Definitions need to fully define each term used in the indicator name to clearly explain what the indicator will measure to improve data quality. If you’re suggesting a custom indicator, explain the justification for its inclusion there.

Field details and definitions include:

· Indicator: The indicator name should be concise and clear about what exactly is being counted or measured, and should spell out acronyms.

· Bureau Indicator:

Standard indicator: from Program Indicator List, including F Standard indicators.

Custom indicator: designed specifically for this project.

· Indicator Type:

Output: Monitors Outputs from project Logic Model, the direct results of activities Outcome: Monitors Outcomes from project Logic Model, the intended changes that result from a project/program activities and outputs

· Definition: The definition clearly defines all terms, and if needed, how to calculate the data collected to measure the change expected. Clarify everything the program design team needs to know to understand how to use this indicator.

· Disaggregation: Defines any breakdown of data reported into sub-categories relevant to analysis. Categories are the specific categories that your data will fit into. For example, Sex is a Disaggregation, and Male/Female are the Categories.

· Report By:

Integer: Reported as a number.

Percentage: Monitors a percentage and is reported as a numerator and denominator.

Qualitative: Reported as text

· Target: Intended result, see Attachment 5, Selecting High Quality Performance Indicators, Setting Targets & Baselines for more guidance. This is the cumulative target for the award entire period of performance.

· Target per Fiscal Year: break out the intended total target by targets per Fiscal Year, which should add up to the total target. This is not entered into MyGrants but is helpful for project monitoring.

· Target Rationale: Describe how, specifically, the target was chosen. For output indicators, note any contextual assumptions and how the budget relates to the target. For outcome indicators, please state causal assumptions or relevant evidence for your target.

· Baseline: Reference point for results at the beginning of performance, see Attachment 5, Selecting High Quality Performance Indicators, Setting Targets & Baselines for more guidance.

· Data Source: Source of data gathered to report on the indicator, be as specific as possible.

· Activities: List the Activities from the Work Plan that contribute to this Indicator

The entries below serve as examples of the workplan and indicator tables. Please modify and add additional lines to the table as needed for your scope of work.

Activity # (Align to Logic Model)

What will this Activity/ Milestone Achieve? When will it start?

(Outputs and short-term outcomes in your logic model) Location Where will this take place?

(Region/ province/ virtual or in-person) Public Event?

(Yes/No) Activity/ Milestone Completion Date (When will this be completed?

MM/YYYY)

Objective 1

Start Date: 06/2024] ☐Virtual ☒In-person – location: Western Region ☐ Yes ☒ No

[Completion: 05/2026]

*The number should be entered in the MyGrants activity name field.

An individual may attend a higher education institution with USG-support in the US, in the individual's country of residence, or in a third country. A country of residence is where the individual resides, regardless or nationality or citizenship. A third country is a country that is neither the individual's country of residence or the US. These designations help USAID missions track investments within and outside the country in which they work.

“Attending” a higher education institution can be for the purpose of long-term training or short-term training. Long-term training is defined as degree-earning coursework that requires six or more months of commitment. Short-term training is a training that is fewer than six months in duration, but typically a one-to-two week classroom-style course that is led by a professional instructor or technical expert with the goal of helping participants acquire new knowledge and skills in their area of employment.

"USG scholarships and financial assistance" are defined as USG-funded, full or partial financial aid provided for a student to further his or her education. This includes financial assistance, waivers, or other support for experiential opportunities such as research assistantships, internships, apprenticeships, etc. This also includes assistance for expenses accrued while furthering one’s education such as housing, health insurance, materials, etc. Such financial aid may be awarded based on a range of criteria (e.g. merit-based; needs-based; career-specific).

"Individuals" are those who have received a scholarship or other form of financial assistance to attend a higher education institution in their own country or another country. This includes financial assistance to US citizens to study in another country and to citizens of other countries to study in the US, their own country, or a third country.

This indicator should report all individuals who received scholarships and financial assistance and attended courses or participated in an academic or training program at a higher education institution during the year being reported, even if some of these individuals may also have been counted in previous years. In other words, if a student is attending a higher education institution with current funding and was counted towards this indicator in a previous fiscal year, the student can be counted towards the indicator again in the current fiscal year.]

Note: you must identify both the disaggregation, in this case Sex, and disaggregation categories, in this case Male/Female]

Target per Fiscal year:

Note: for AC use only, not for entry to MyGrants

[FY25: 100

FY26: 100]

Example 2: Given IP expects to train 90 people, IP anticipates holding 3 training sessions assuming there is sufficient demand to fill the 30-person classroom.

Describe how, specifically, the target was chosen. For output indicators, note any indicator-specific contextual assumptions and, ideally, how the budget relates to the target. For outcome indicators, please state causal assumptions or relevant evidence for your target.]

Describe where the data used to report against this indicator is from, for example, existing administrative records, a participant feedback survey, etc.]

Note which Activities from the Work Plan, above, contribute to this performance indicator.]

Planned Evaluation(s): This section describes any formal evaluations planned over the project period. Evaluations differ from routine monitoring in that they systematically collect or analyze information about the program, project, or process at a defined point in time as a basis for making judgements, improving effectiveness, and informing decisions, rather than tracking ongoing progress against indicators. If proposing an evaluation, include the responsible party, estimated budget, timing, evaluation type, purpose, and key stakeholders.

Attachment 2: Theory of Change and Logic Model Template

Logic Model Template

Problem Statement

[State the main problem statement. What problems or issues does the project attempt to solve of address?]

Main Project Goal

[State the main project goal. The following Project Objectives will feed into this Main Project Goal, which will in turn support the overarching IR.]

Project Objective 1: [State the project objective here and divide the logic model by each objective]

(immediately after activity) Short-Term Outcomes (0-6 months after activity) Long-Term Outcomes (6+ months after activity)

[Examples:

Staff

Money

Materials

Equipment

Partners] What activities will be conducted to achieve project objectives? Activities are what you do.

[Examples:

Conduct workshops, meetings.

Provide trainings.

Develop curriculum of applications.

Deliver services.] What will be produced or delivered and who will participate in activities? Outputs are what you produce/create/deliver.

[Examples:

# Products created.

# Resources developed.

# Participants trained.

# Services delivered.

# Beneficiaries reached.] What changes will happen immediately after to six months after an activity has been implemented? Short-Term Outcomes are the results of your outputs.

[Examples:

Increased awareness or knowledge.

Improved skills.

Increased access to services or products.

Reported change.] What long-term changes will occur in the social, cultural, political, or economic context? These could be changes in behaviors, actions, or conditions in the country. Long-Term Outcomes are relatively enduring results of your outputs.

[Examples:

Change in behavior.

Adoption of best practices.

Sustained access to services or applications]

Project Objective 2: [State secondary project objective here. Add additional Project Objective sections as necessary to report on all project objectives]

(immediately after activity) Short-Term Outcomes (0-6 months after activity) Long-Term Outcomes (6+ months after activity)

What factors need to occur, or which stakeholder or grantees need to be involved, for the project to be successful? How and why will proposed activities lead to proposed outputs, and how will outputs lead to outcomes?

[Country’s political and economic condition; Skills and knowledge level of intended beneficiary; Implementer’s organizational capacity; Level of engagement by stakeholder] What factors that are not in the control of the project- e.g. changes in government policies or the political situation- could affect expected activities, outputs, and outcomes?

[Change in government policies, such as NGO restrictions; Change in political situation, such as coup or civil unrest; Involvement from other donors and implementers]

Theory of Change Statement

Describes how and why a program is expected to achieve its stated outcomes using if-then language.

If [insert project outcomes] then [insert project objectives], because [insert rationale/ research/ reasoning for this intervention to achieve your objectives]

Note: This is a recommended template; any format is acceptable as long as the information above is included.

Attachment 3 – Selecting High Quality Performance Indicators, Setting Targets & Baselines Key Principles for Choosing Indicators The development or selection of indicators should be based on a logic model that links the goals and objectives of a project with its activities, outputs, and outcomes. Indicators monitor progress towards achieving the project outputs, outcomes, and objectives.

· Relevant: It must align with your project's goals, objectives, and the output or outcome it’s monitoring. If improving education is the goal, focus on academic indicators rather than unrelated measures like event attendance.

· Clearly Defined: Specify exactly what you're measuring. For instance, if your project aims to improve education, an indicator could be “the number of students who improve their math test scores.”

· Measurable: Ensure it can be observed, counted, analyzed, or tested. For the above example, test scores can be measured.

· Achievable: The indicator target should be realistic within your project's timeframe and budget. Example: Aim for a 10% improvement in test scores within a year, rather than 100%.

Types of Indicators

1. Output Indicators: Output indicators measure the direct results of activities, like “number of textbooks distributed," or “number of training attendees”, and even “number of vaccine doses delivered.”

2. Outcome Indicators: measure the changes that result from a project/program. For example, “Percent of students graduating,” or “Percent of children immunized.” A long-term outcome might be, “Reduction in prevalence of disease in the target community.”

Setting Indicator Targets A target is a specific amount of achievement/result that a project aims to achieve/reach. These targets are essential for measuring progress towards desired outcomes and implementers must establish targets for each performance indicator prior to reporting actual data. Targets must be realistic and account for the project context, timeframe, and available budget. Setting targets should occur through negotiations with all program team members – the Implementer, Programs, Budgets, Grants, and MEL.

TIP: For both output and outcome indicators, it is advisable to break down the targets into more detailed components. For example, if you’re aiming to improve literacy rates in a community, you might start by reviewing the current literacy rate of the community, success rates of past literacy projects, consult educational experts, and look at studies of similar literacy programs.

· For Output indicators, consider a project aimed at improving environmental awareness among local government officials. On the demand side, you’d need to figure out how many officials are actually interested in and able to attend such training sessions. On the supply side, think about how much money NEA/AC can provide, how much it costs to train one official, and whether the organization doing the training can effectively use the fund to organize and conduct these sessions. It’s also important to consider how experienced the organization is in environmental training and how likely they are to successfully improve awareness based on their track record.

· For outcome indicators setting targets for a project is more complex because many factors, not just the project activities themselves, can influence whether the expected outcomes are achieved. These factors often include external elements beyond the project’s control. To make target setting for outcome indicators more manageable consider these strategies:

· Look at historical data or contextual data for the same or similar indicators to understand past trends and baselines.

· Consult an expert in the projects field to gain insight based on their experience and knowledge.

· Review available research from similar projects or programs to learn from their successes and challenges.

Finally, set sub-targets by Fiscal Year, if possible, to inform the Performance Plan and Report and to track progress towards targets per year of the award.

Setting the Indicator Baseline

1. What Is a Baseline? The starting point of your indicator before the project begins. It's essential for tracking progress. Throughout an award lifecycle, the baseline should always reflect the beginning of the period of performance, even if the project end date is extended.

2. When to Set Baseline at Zero? For new outputs directly resulting from your project, like “number of students attending after-school tutoring sessions,” start at zero. New outcome indicators may also be set at zero. If you're building on existing conditions, such as improving test scores, use pre-existing data to establish your baseline if it’s available and relevant.

Updating Indicators During a Project Period of Performance:

Before modifying or creating new indicators for an active project, the NEA/AC MEL team needs to assess the tradeoff carefully and analyze whether the changes will benefit or hinder the project from monitoring progress towards achieving its goal. Modifying targets or indicators (e.g., removing an indicator or lowering a target because the project did not achieve the target) within the final months of a project is discouraged from the standpoint of ensuring accountability and accuracy (i.e., a project’s inability to meet its targets is actionable information that could inform future project design and implementation decisions).

Updating or removing an indicator may be justified if there was a major change in the project’s activities, a better monitoring method was developed, or existing indicators were found to not capture output/ outcome data accurately. If modifying the RMP to include new or modified indicators, the NEA/AC MEL team, in consultation with the implementer, programs, and grants team, should consider:

· Determine if the indicator should be updated or retired: If an indicator definition is being updated to provide clarity that was previously lacking, and the indicator continues to measure the same thing as before, the indicator can be updated. However, if an indicator is updated to meaningfully change what it is measuring, it should be retired and no longer reported on, and any new indicators should be added.

· Retaining one or two core outcome-level indicators that track higher-level results: Core outcome-level indicators can act as a guiding “north star" to ensure the project stays on the intended course, to preserve trend data, and to maintain accountability on whether the project is achieving its goal. Any changes to the work plan can be compared to the outcome-level indicators to assess if the changes have made the project more effective and attained the results it set out to achieve.

· Consideration of establishing a proxy indicator as an indirect measure: there will be instances when data will not be available for the most suitable indicators of a particular result. In these situations, the NEA/AC MEL team should consider a proxy indicator which is a less direct way of measuring against a result. For instance, where tests of exchange students’ achievement have not been systematically applied to assess education quality, a proxy measure might focus on the student completion rates. The assumption is that if there is a high completion rate, then it may be reasonable to assume improvement in education quality. Proxy indicators must be carefully assigned; the use of inadequate proxy indicators can threaten the validity of the data collected (for instance, students’ self-reported levels of achievements in mathematics won’t tell us anything about the quality of education).

Example: updating an indicator target during the project period of performance

Target: 100 Explanation: the initial project planned to train 100 students over a two-year period. At the beginning of the project’s period of performance, no students had been trained.

Baseline: 0 Target: 100 Explanation: the initial project planned to train 100 students over a two-year period. At the beginning of the project’s period of performance, no students had been trained. The project requested a NCE to complete the initial target of 100 students trained due to unforeseen delays in the final training cohort.

Baseline: 0 Target: 150 Explanation: the initial project planned to train 100 students over a two-year period. At the beginning of the project’s period of performance, no students had been trained. Additional funds became available to support this project, leading to plans for an additional year of training, with a target of 150 students trained.

Attachment 4 – Most Significant Change Indicator Guidance

For use if the MSC Indicator is selected for use in the RMP.

Most Significant Change (MSC) is a participatory monitoring and evaluation technique focused on documenting the qualitative effects of social change interventions. Implementers describe the change(s) that occurred and the reasons why they consider such changes to be significant. The technique is participatory in that it requires the involvement of project funders, project implementers and project stakeholders; these actors can be involved in both the collection and analysis of outcome data. It allows project stakeholders at all levels to have regular discussions about what observed changes are significant and why.

MSC is a flexible technique that can be adapted to a variety of programming contexts. It is particularly suited to complex environments where conventional quantitative measures may be inadequate to assess project effects, or where outcomes are diverse in scope and there is a potential for unexpected effects or consequences. MSC is particularly useful for complex or innovative activities where cause and effect relationships are not well understood, and for activities that seek to influence policy or social change.

Grantees (or the funder) will include MSC as a qualitative outcome indicator in the RMP. The following is the information that should be included in proposed RMPs, which will be finalized in coordination with NEA/AC’s MEL Specialist and then uploaded into MyGrants, as appropriate:

· Indicator Code: MSC-1

· Bureau Indicator: Most Significant Change

· Indicator Definition: Total number of significant change stories collected from project stakeholders that tell the story of significant changes and outcomes that are directly or indirectly linked with the NEA-funded projects. Data collection activities with stakeholders will begin approximately six months after the start of their participation/involvement in the project and continue when appropriate within the project until the conclusion of the award. The data collection process and reporting templates will be shared by the NEA MEL POC. During the project lifecycle at minimum one significant change story with a maximum of four will be submitted each project year.

· Indicator type: Qualitative

· Indicator Level: Outcome

· Target Number: [total number of years the project spans]

· Frequency of Reporting: 1-4 per project year

· Data Source: Implementer records, interviews, or discussions with direct or indirect project stakeholders