5a. SGWI Logic Model, MEL Plan, and PITT Instructions 2023 (final).docx

S/GWI– Monitoring, Evaluation, and Learning Plan Guidance

Project/Activity Monitoring, Evaluation, and Learning Plans

Monitoring, Evaluation, and Learning (MEL) is an organized process for collecting and analyzing information to determine if an activity is achieving its goals and objectives, whether and how implementation should be adjusted, and what can be learned to strengthen future projects or activities. This is done through creating, utilizing, and reporting on a Monitoring, Evaluation, and Learning (MEL) Plan (see Attachment 5b).

Each MEL Plan should be aligned to the theory of change and logical or results framework in the overall project design, meaning that goals and objectives are supported by outcomes, outputs, and activities with associated indicators and milestones. Targets and actual progress for indicators and milestones need to be reported by quarter as possible, and always by fiscal year. That means that for the final quarterly report of each fiscal year (July-September report), all indicators should have the cumulative total for the fiscal year and the target for the coming fiscal year in the reports. Milestones may also be used to signal progress towards an intended result. All indicators, whether at the outcome or output levels, should be accompanied by baselines, targets, disaggregation, data sources, and the frequency of data collection. S/GWI has provided a sample MEL Plan template for convenience. Applicants are not required to use the attached template, but if using their own, they should at least include all the components described here. Each component is described in detail below.

Logic Models and MEL Plan Connections

Goals and Objective framework, called a Logic Model, matched to MEL and Outcomes and Outputs: Please use the same goals and objective framework listed in the Project Narrative to show linkages to outcomes and outputs measured in the MEL plan. Applicants may also choose to include their activities in the outline as they are the connection between goals and objectives and then the outputs and outcomes measured by MEL. Since S/GWI encourages community-led project design, we recognize that project activities may not yet have been developed at the initial proposal stage.

When connecting the Logic Model to the MEL Plan, which means outcomes and outputs, it can be helpful to think of outputs as the deliverables of your project’s activities (one training, for example, is the deliverable of a series of activities that include developing a training curriculum, securing a venue, and inviting participants). Together, multiple outputs build towards a desired outcome (three trainings, for example, can lead to an increase in participants’ knowledge). Outcomes fall between outputs and your goal. Oftentimes, a goal is reached through the achievement of several outcomes. There should be at least one (1) outcome indicator for each objective. Implementing partners are expected to submit a performance indicator reference sheet (PIRS) for outcome indicators.

Indicators: Indicators are used to track progress towards objectives and the goal. Indicators should be quantifiable and clearly defined, meaning that there should be a clear unit of measurement associated with the change to be measured. Qualitative indicators are acceptable if they can accurately and reliably show change in a particular phenomenon or attribute. While applicants are expected and encouraged to develop project-specific outcome and output indicators, where possible, please also integrate relevant standard foreign assistance indicators (Attachment 5c) to help S/GWI tell the story of its impact across different programs.

While S/GWI welcomes some output indicators, we are most interested in outcomes, sometimes called results, indicators. At the same time, we understand that cost and time must be balanced when determining what indicators to collect. The deliverables that combine to achieve a project’s results, outputs, and their indicators can only show whether a deliverable has been produced. They do not help us understand whether that deliverable contributed to meaningful change (for example, an output indicator counting the number of trainings cannot tell us whether its participants will participate in an election). Rather, output indicators tell us whether a project is on track and on time.

Outcome indicators measure the change that has taken place as a result of the project’s deliverables (for example, an increase in the percentage of women who vote in the election). Whereas output indicators answer the question, “has this happened?”, outcome indicators answer the question, “did anything change as a result of these things happening?” Each project objective must be accompanied by at least one outcome indicator.

Baselines are indicator measurements collected before or at the start of a project. They help inform targets and serve as a benchmark for measuring subsequent progress (e.g., percent of women who already vote each year). If 15% women already vote in each municipal election, then the baseline for “women’s voter participation” should be 15%. For some S/GWI projects, the baseline might be 0 or it may be determined by a baseline study conducted during project inception.

Targets are reasonable estimates of the outputs or outcomes to be achieved by the project over its period of performance and provided by quarter and fiscal year. Targets are estimates. Performance may be higher or lower than the target. For any baselines or targets expressed as percentages, please include information on how the percentage was estimated, as demonstrated in the examples. Because programs are expected to adapt to feedback and progress, S/GWI expects targets to shift over time, and will request discussion and documentation of how/why those shifts need to happen.

Performance Indicator Reference Sheets (PIRS) All custom outcome indicators and standard FA indicators are required to have a PIRS submitted with the final MEL plan for Grant Officer Representative (GOR) approval during start-up (see Attachment 5d). At a minimum, the PIRS must include:

· Project Outcome

· Indicator or Milestone Title

· Definition

· Linkage to Project Goals and Objectives

· Reporting Type

· Use of Indicator or Milestone

· Reporting Frequency

· Data Sources

· Indicator Owner(s)

· Data disaggregates

· Documentation methodology

There are global, fixed, standard PIRS for the standard FA indicators,[footnoteRef:1] that only need to be added to with how the implementing partner will implement that standard FA PIRS. [1: https://www.state.gov/foreign-assistance-resource-library/ ]

Data Sources are where the project will find information on its indicator reporting. Data sources should be easy to use for both the participant and the person responsible for analysing the data. The sources, processes, frequency, disaggregation, documentation methodology, and people responsible should be documented in a performance indicator reference sheet (PIRS) that can be developed during start-up. Suggested data sources include, but are not limited to:

· Pre and post tests

· Pre and post intervention surveys/questionnaires

· Meeting minutes

· Attendance sheets

· Site visit checklists

· Interviews

· Focus groups

· Secondary Data: data that already exists (e.g.: Government reports, census information)

· Program reports

Data Disaggregation separates data into its component parts to look at how specific subgroups perform. Since data can be disaggregated in an infinite number of ways, it is important to disaggregate only by the information that you need to know. S/GWI requires all data to be disaggregated by sex (M/F/X) and age (at least adult/child, with more disaggregates as applicable to project design), at a minimum. Disaggregated data helps us understand the different ways in which the project impacts and is impacted by men, women, boys, and girls. Data may need to be divided into additional categories (e.g., region, vocation, type of organization, etc.) to meet the needs of the program.

Frequency of Data Collection establishes when data will be collected (e.g., monthly, quarterly, annually, or at project start or end). Note that this is not necessarily the frequency of the activity (e.g., literacy classes delivered daily), but rather the frequency by which data will be collected. Consider what timing is feasible for each indicator.

Learning Plan Data and monitoring is only worth the effort if it can improve the effectiveness of the work. Please describe how the learning that takes place throughout implementation will be incorporated to strengthen the project’s effectiveness and avoid unintended consequences. This should include the point person or people responsible for overseeing and reporting this this learning. If awarded, S/GWI encourages awardees to provide success stories and anecdotal or other qualitative evidence of project impact in the quarterly progress reports, in addition to documenting indicator progress in the MEL plan.

This plan should include annual reflect, consult, and adapt sessions with local implementing partners to: 1) review progress and challenges against goals and objectives, 2) update targets and analyze data quality for the MEL plan, 3) update and resubmit the gender and inclusion analysis (as it will relate to overall implementation), and 4) analyze and report against progress implementing in coordination with other implementers and donors consistent with S/GWI principles (i.e. survivor-centered, inclusive, etc.).

Data Security The MEL plan should include project data governance and custody protocol, as well as staff training to ensure that sensitive data is handled ethically, safely, and appropriately to ensure accurate reporting and the safety and security of stakeholders.

Evaluations Many S/GWI grants require independent evaluations. An independent evaluation must be (1) conducted by a third-party external evaluator or conducted by the recipient if those responsible for the evaluation do not report to any program staff or managers, and (2) budgeted and scoped to provide insight and analysis of the work. Please explain how the evaluation will be independent and how its findings will be disseminated, if appropriate, to strengthen the applicant’s and/or the field’s evidence base.

S/GWI recognizes that it may be difficult for some applicants to design results-oriented MEL plans. While we provide some flexibility, where appropriate, we encourage applicants to be as ambitious, comprehensive, and creative as possible.