Defining M&E Terms.docx

Monitoring and Evaluation (M&E) at the Department of State: Defining Terms Problem Statement: This is a clear and concise description of the issue, challenge, or need the program seeks to address and serves as the program focus. Developing a problem statement provides the basis and reasons for implementing a program.

To develop a problem statement, answer: who, what, where, when, why, and how:

1. Identify “Who”

· Understand the demographics of the possible beneficiaries or customers

· Understand the key stakeholders engaged in or who may be affected by the program’s activities

2. Identify “What”

· Understand the scope and subject matter of the issue

· Understand the extent, or the size and scale of the issue

· Understand the severity, or the gravity, and urgency of the issue

3. Identify “Where”

· Describe the geographic or specific locations where the issue takes place

· Understand the location of the issue or problem

4. Identify “When”

· Understand the timing of the issue or problem, if applicable

· Understand contextual factors that affect when it occurs

5. Identify “Why”

· Understand the cultural impact surrounding the issue or problem: politics, beliefs, customs, norms, power structures, etc.

· Understand the underlying root causes of the issue or problem

· Understand perpetuating factors that give rise to the issue or problem

6. Identify “How”

· Review the customers and stakeholders identified in “who” and discuss more specifically how this issue affects this population

· Review gender analysis (if conducted)

7. Develop the Problem Statement: the previous six answers combine to help form the problem statement, which should be a clear statement of the issue or problem the program would like to address.

A Theory of Change (TOC) is a brief statement that ties your logic model together by summarizing why, based on available evidence and consideration of other possible paths, and the changes described in your logic model that are expected to occur. A TOC is also an evolving set of theories about how and why a specific program will work that can be tested and improved over time. TOCs often include narrative statements that describe why it is believed one step will lead to another (e.g., what is the basis for asserting increased knowledge leads to behavior change in this program context?). A TOC can be made more robust when based on literature, research findings, past evaluations, policy, or leading practices such as anything uncovered in your situational analysis. TOCs may be established theories pulled from existing sources; they can be customized based on evidence related to your particular program context, or a combination of both. Consider the following when developing a TOC:

· Changes: What needs to occur to move from the problem that currently exists, to achieving program goals and objectives?

· Theory of Change Hypotheses: Overarching statements of how you believe change will occur to solve the problem. These can be written as “If____, then____ because ______” statements to link existing conditions and desired changes.

· Source/Evidence: Any source or evidence that supports these theories. This could include examination of well-established theories, previous program evaluations, discussions with stakeholders, input from program beneficiaries or customers, and research or literature review. If there is no current evidence to support the theory or assumption, note that, but continue and consider whether support for your theory could be explored as part of your monitoring and evaluation plan.

Consideration of these factors allows the program team to summarize why, based on available evidence and consideration of other possible paths, the particular changes described and predicted in your logic model are expected to occur. The theory of change can be noted on your logic model document, or separately, and should be reviewed periodically to determine if it should be modified as any new information becomes available.

The terms goals and objectives are often used interchangeably, as both point toward the anticipated results of the program, but each serves a different purpose. Goals are broad and indicate overarching aims that the program contributes to, but they may not necessarily be achieved within the timeframe of the program. Objectives are narrower, more specific, and should be realized within the program timeframe. Program objectives should articulate desired results that will help accomplish program goals. There are usually two to five program objectives per program goal. Note that goals and objectives can be created separately, or as part of your logic model Program Goal(s) explain the overall intent or purpose of the program to which program objectives and subsequent activities are expected to contribute. Goal(s) focus on the desired outcomes. Program goal(s) should be:

· Broad and visionary: Program goals are ambitious in nature but also realistic. They should be specific enough to clearly communicate program aims, express future direction and vision by using active or directional verbs, and avoid overly technical terms or acronyms.

· Long-term: Program goals may extend beyond the timeframe of the program.

· Not program resource-dependent: Program goals should not be limited by program resources because they represent a broader desired end state to which the program is contributing.

· Strategy alignment: Program goals should align with and advance existing strategies.

Objectives are the highest level result the program can affect or achieve towards accomplishment of the program goal(s). They are statements of the condition(s) or state(s) the program is expected to achieve within the timeframe and resources of the program. To enable sound performance management, objectives should ideally incorporate SMART principles: Specific, Measurable, Achievable, Relevant, and Time-bound.

· Specific: Describes the desired result in concrete and clear terms such that anyone reading it should interpret it in the same way

· Measurable: Can be evaluated and/or assessed against some standard such that it is possible to know when the objective is met

· Achievable: Attainable within allotted time and resources

· Relevant: Linked to achieving the program goals

· Time-Bound: If applicable, describes when the objective will be complete and any check points relevant to program goals, activities, or resources

A logic model is a tool to systematically document and visually represent program investments, activities, desired results, and the relationship between them. The process of creating a logic model is one of the most critical steps in both program design as well as performance management. It allows the program team and key stakeholders to work through how and why they think the program will work to achieve established goals and objectives, and it becomes the basis for designing monitoring and evaluation plans. Establishing what the program is expected to achieve sets the foundation for what to monitor and evaluate progress against.

The components of a logic model include:

· Short- and Long-Term Outcomes: what we achieve

· Outputs: what we get

· Activities: what we do

· Inputs: what we invest Using the template provided, you can see that each component from left to right is intended to build on or result from the previous one. Outcomes are the intermediate and long-term results of the outputs. Outputs are the results of the activities, and activities are based on the planned program inputs. When initially creating a logic model, however, it is most helpful to start from right to left whereby stakeholders first discuss and agree upon the intended outcomes so the other components lead to these desired long-term results.

· Long-term outcomes describe system- or societal-level results, and can focus on behavior, normative, and policy changes the program seeks to achieve. Though often not realized for years after program implementation, long-term outcomes are based on what success would look like if the program goal(s) were ultimately achieved.

· Short-term outcomes describe the immediate effects of the program and often focus on changes to the knowledge and attitudes of the program’s beneficiaries or customers, such as a change in attitude as a result of an awareness campaign or a change in behavior due to a training program. The short-term outcomes of the program should be linked to the program objectives.

· Outputs are the direct, tangible results of program activities. They are often easy to count, such as number of people trained or number of awareness campaigns conducted, and generally measure the amount or quantity of the activities or beneficiaries/customers.

· Activities are the actions or events undertaken by the program or partners to ultimately produce desired outcomes. Activities identify measurable action steps to achieve the program’s goals or objectives, such as conducting outreach, providing training, or developing a communication campaign. They should be feasible from an economic, social, and political perspective, and should have a clear link to addressing the intended issue or challenge identified in the problem statement.

Program assumptions describe the conditions that need to exist so that one step in the logic model can succeed and lead to the next step. These could include conditions over which program implementers may or may not have control.

SENSITIVE BUT UNCLASSIFIED

SENSITIVE BUT UNCLASSIFIED

SENSITIVE BUT UNCLASSIFIED