SGWI Monitoring and Evaluation Primer.pdf

Monitoring and Evaluation Plan Primer for S/GWI Grantees

All project proposals being considered for S/GWI funding must include a comprehensive M&E plan. This plan is one of the key criteria in determining the competitiveness of a project proposal.

I. What is a monitoring and evaluation plan?

A monitoring and evaluation plan (M&E plan), sometimes also referred to as a performance monitoring or performance management plan, is a systematic and objective approach or process for monitoring a project’s performance in meeting its objectives over time.

The plan consists of indicators with baselines and targets; means for tracking critical assumptions; and plans for managing the data collection process and for regular collection of data. Evaluations should be scheduled and carried out throughout the course of the project.

Development of an M&E plan is integral to the planning of a project’s design. In fact, a rigorous M&E plan can be an effective tool for formulating a coherent and well-designed project proposal, both in revealing assumptions and exposing gaps in project planning.

As an implementer modifies its project design, it is important to incorporate those changes into the M&E plan and vice versa.

II. Why is it important?

As a planning document, an M&E plan forces an implementer to think systematically through each of the project components. Preparing the M&E plan is also a way to assess and document the specific linkages between project activities, the intended results of the projects, and the indicators that will be used to measure the achievement of these results. Careful attention to the M&E plan in the earliest phases of project planning and design ensures that the project funding is aimed at creating impact and not simply outputs.

During project execution, a well-designed plan helps an implementer keep track of the project’s progress and make adjustments if necessary. It is also a valuable tool for implementers and donors in assessing the effectiveness and impact of a project, generating credible information that contributes to learning, improved performance, and accountability. For the donor, if a project has proven results, the M&E plan is a useful method for showcasing the project’s success and documenting the implementer’s track record. If the project is less than successful, the M&E plan can help to identify specific weaknesses and make adjustments to improve project performance. In either case, the M&E plan is useful for informing the donor’s decisions about future budget allocations and programming.

III. Goals of a monitoring and evaluation plan

An M&E plan focuses on the performance of a project and examines its implementation plan, activities, outputs, and results. A project is defined as an individually planned undertaking designed to achieve specific objectives within a given budget and time frame.

Monitoring and evaluation play distinct, but complementary, roles in assessing project performance. Monitoring entails ongoing oversight of the performance of activities through collection and analysis of routine measurements; it is used to inform managers about the progress of an ongoing intervention or project and to detect problems that may be able to be addressed through corrective actions. Evaluations of projects are only conducted periodically and go beyond activities to analyzing results—both intended and unintended—and focus on changes in the target populations or systems the project was designed to affect. Monitoring data is used in evaluations, but is often complimented with qualitative and more in-depth information from various surveys and other data.

Most of the information for monitoring and evaluation can be gathered through reviews of project documents; interviews with project staff, host country officials, and other stakeholders; and developing and conducting surveys, focus group discussions, and meetings with participants and other beneficiaries.

A solid M&E plan generally includes a mid-term and final evaluation. The mid-term evaluation can measure the interim progress of a project and identify areas that could be modified to improve performance and impact. The final evaluation can not only assesses the project’s overall results, but can also provide an analysis on lessons learned and make future recommendations.

IV. Elements of a monitoring and evaluation plan

A. Goals and Objectives

The difference between goals and objectives

The terms goals and objectives are often used interchangeably, but there are distinct differences between the two, mainly in terms of their level of concreteness. Both terms imply the target desired as a result of one’s efforts. However, goals are less structured, whereas objectives are concrete and viewed as targets under the general goal. Also, because goals tend to be lofty in nature, it is not possible to measure the accomplishment of one’s goals. But one goes about measuring progress towards a goal by setting tangible objectives with clear timelines, which are possible to measure.

Below is a table delineating the key differences between goals and objectives.

Goals Objectives

Broad and visionary in nature Narrow and time bound General intentions Precise

Not resource dependent Resource dependent Intangible Tangible Abstract Concrete

Project activities contribute Achievement of objectives is to achievement of goals (can’t be validated) attributable to project activities (can be validated)

Goal The overall intent of the project to which project activities are expected to contribute.

• Strong: “Women play an active role in making decisions that affect their communities” (describes a goal to which project activities will contribute, but that may also be influenced by other variables such as political events, cultural factors, or the work of other organizations)

• Weak: “Women trained by the project demonstrate increased knowledge of the decision-making mechanisms of the local council.” (goal is too narrow and specific to project activities)

Objectives Define the change that a project intends to achieve and for which the recipient is held accountable.

: S/GWI projects usually have two to three objectives (two is the minimum)

Objectives should be:

• Unidimensional (one result per objective) o Strong: “More women candidates run for public office.”

o Weak: “More women are trained, run for, and are elected to public office.”

• Stated as a “done deal” or a desired end state o Strong: “Independent media organizations are financially sustainable.”

o Weak: “Promote financial sustainability of media organizations.”

• Describes an achievement – not an activity or process o Strong: “Family law reform adopted by Parliament.”

o Weak: “Policy reform dialogue sessions held.”

B. Activities

Activities: For S/GWI projects, there should be at least two activities associated with each objective.

Activities are the major actions carried out with grant funds during the performance period.

They are directly linked to achievement of project objectives and are sufficient to reach those objectives.

Activities should be well-defined in terms of audience, location, duration and cost:

o Strong: “Conduct two-day training program by Prof. Djelloul at the Marrakesh Civic Education Center on municipal elections monitoring process for 25 political science students from the University of Marrakesh. Training must be completed 30 days before June 19 elections.”

o Weak: “Election monitoring training.”

NOTE: A good activity description designates the person responsible for implementing, defines the activity location, specifies duration of the activity, indicates the number of participants, sets deadlines to be respected, and provides details to justify budget requested.

C. Indicators

Performance indicators are used to observe progress and measure actual results compared with expected results. Performance indicators should be linked to the project’s strategic objectives and focus on outputs and results.

In differentiating between outputs and results, it can be useful to think of results as developmentally significant changes that impact the beneficiaries, while outputs are lower-level steps that are not developmentally significant but are essential in achieving results. Outputs may impact the beneficiaries but in a more limited way than outcomes. The link between outputs and results should always be direct. For example, the activity could be training community journalists on investigative journalism through a series of workshops. One output could be the number of community journalists who are trained. The results could be the new skills that participants’ develop as a result of the workshops.

Measures should be in quantifiable terms and clearly defined. Qualitative indicators, though more rare than quantitative indicators, are acceptable if they provide a reliable means to measure a particular phenomenon or attribute.

A good guide for determining performance measures is the acronym SMART. Indicators should be:

• Specific

• Measurable

• Achievable

• Realistic

• Time-framed

Performance Indicators: Each objective listed must have at least one results indicator

Indicators track progress towards expected results. S/GWI projects track two types of indicators:

Results indicators (also called outcome indicators) focus on change and are a measure of the extent to which a project objective is being achieved (higher level). Results indicators should answer the question, “How will we know achievement when we see it?”

o Number of women who demonstrate increased knowledge of HIV prevention measures (change in knowledge)

Output indicators do not measure change. They instead measure the products or outcomes of project activities (lower level).

o Number of training sessions held (product of activity, no change)

Besides defining the performance indicators in the M&E plan, it is also important to set baselines and targets for each indicator and describe the methodology for measuring the performance indicators (e.g., pre- and post-test surveys, focus groups, interviews, etc.), including the data source (e.g., training sign-in sheets, website hits, survey data) and the frequency of measurement (e.g., after each workshop, quarterly, annually).

Baselines and Targets

: Baselines and targets must be specified for each indicator included in the M&E plan.

Baseline measures for an indicator are collected before or at the start of a project and provide a basis for planning and/or assessing subsequent progress and impact.

Targets are reasonable estimates of the outputs or results to be achieved by the project over its period of performance. Targets are estimates

. Performance may be higher or lower than the target. If there were any circumstances that contributed to stronger or weaker than expected performance, recipients should explain these circumstances in the narrative section of their S/GWI quarterly report.

Example

Indicator: Number of women initiating community development projects after project training.

Baseline: 0 Target: 10 Result Indicator at end of project: 5

V. Drafting an M&E Plan

Each M&E plan should contain specific output indicators and results indicators with baselines and targets, data source, and frequency of data collection. However, there is not a standard template for the M&E plan that each S/GWI recipient must follow (e.g., each objective does not need to a predefined number of results indicators, activities, or output indicators).

S/GWI wants to afford recipients the flexibility to design an M&E plan that is customized to the specifics of the project. S/GWI recognizes that sometimes it may be difficult for recipients to design truly results-oriented M&E plans, but we encourage recipients to develop an M&E plan that is as comprehensive, ambitious, and creative as possible.

The M&E plan is meant to detail how the outputs and outcomes of project activities will be measured quantitatively. Yet, sometimes the results and impact of a project are more easily conveyed qualitatively. Recipients can describe how project impact will be assessed qualitatively at the end of the M&E plan. S/GWI encourages recipients to provide success stories and anecdotal or other qualitative evidence of project impact in the quarterly progress reports, as well as showing how well the project is meeting the targets set in the M&E plan.