Program-Design-and-Performance-Management-Toolkit (1).pdf

Program Design and

Performance Management

Toolkit

TABLE OF CONTENTS

Program Design And Performance Management Toolkit Use And Navigation

Toolkit Structure And Navigation

Section 1: Align Programs To Advance Existing Strategies

Section 2: Conduct A Situational Analysis

Step 2.1: Ways To Conduct A Situational Analysis:

Step 2.2: Types Of Situational Analyses:

Section 3: Design A Program

Step 3.1: Develop The Problem Or Needs Statement

Step 3.2: Develop Program Goals And Objectives

Step 3.3: Develop A Program Logic Model And Theory Of Change

Section 4: Manage Performance

Step 4.1: Understand Monitoring And Evaluation

Step 4.2: Develop Indicators

Step 4.3 Complete The Monitoring Plan And Indicator Tracking Table

Step 4.4: Plan For Evaluation

Step 4.5: Using The Toolkit With External Implementers

Section 5: Analyze, Learn, Act

Step 5.1: Analyze And Learn From Data

Step 5.2: Act

Step 5.3: Organize Information

Appendix

Appendix A: Design A Project

Appendix B: Design A Process

Appendix C: Blank Tool Templates

Appendix D: Data Collection Methods

Appendix E: Case Example 2, Embassy Disaster Preparedness

Appendix F: Key Terms

Appendix G: Works Cited

PROGRAM DESIGN AND PERFORMANCE MANAGEMENT

TOOLKIT USE AND NAVIGATION

TOOLKIT PURPOSE AND USE

Sound program design and performance management is the basis for the effective and efficient use of State Department resources to achieve our strategic goals. Program design details how we plan to achieve the strategic goals for a sector, region or country, and performance management enables us to assess the extent to which our efforts are working and why. Steps and tools provided in the Program Design and Performance Management (PD/PM) Toolkit can be applied to new and ongoing State Department programs, projects, and processes across diplomatic engagement, administration, and foreign assistance. This Toolkit is designed to be applicable to all bureaus, offices, and posts and the templates can be customized to fit your needs.

The PD/PM Toolkit will help bureaus, offices, and posts design their activities in a way that effectively advances broader strategic goals, utilizes useful monitoring and evaluation, and enables learning to occur that can inform future decisions. It can be used as a manual for creating and executing a new program, or can be employed mid-program to determine if the effort is on track.

The PD/PM Toolkit outlines the major steps of program design and performance management along with suggested tools and templates for completing each step. Completing each step will provide bureaus, offices, and posts with a program summary as well as a performance management plan that can be used for knowledge management, accountability, and to communicate about the program internally to leadership and staff and externally to partners, implementers, or other stakeholders.

The PD/PM Toolkit was jointly developed by the Office of U.S. Foreign Assistance Resources (F) and the Bureau of Budget and Planning (BP). Because some State programs are coordinated with USAID, to encourage congruence and a common lexicon, concepts from the USAID Automated Directives System (ADS) are incorporated where possible. It replaces the Department’s Performance Management Guidebook, Program and Project Management Guidebook, and Project Design Guidebook.

TOOLKIT STRUCTURE AND NAVIGATION

The PD/PM Toolkit is divided into five sections, and within each section are multiple steps. It provides an explanation of each step, a template for implementing that step, and a fictional program case example to illustrate a completed template. Blank templates, a second management focused case example, and additional information about project and processes are included as appendices. While each section of the PD/PM Toolkit builds upon the previous section, they can also be used in a stand-alone fashion to support on-going programs.

The following icons are used throughout the PD/PM Toolkit:

More about the Case Example: The case example throughout PD/PM Toolkit follows the fictional South Bureau as it develops a new clean energy program in the fictional country of Freedonia. For each concept and tool introduced, the Toolkit provides a completed case example using this fictional program, which is intended to illustrate the types of information you could include in the blank tool templates as they follow the instructions for each section and step. Another case example, which can be found in Appendix E, highlights program design for management/infrastructure efforts. This case example follows the fictional Management office in Embassy Capital City as it develops a disaster preparedness program for State Department personnel.

Identifying Program, Project, or Process: The PD/PM Toolkit can be used by managers of programs, projects, and processes. In Section 3 of the Toolkit, project and process managers may want to turn to additional resources outside of the main Toolkit before returning to Sections 4 and 5. The graphic map titled “Am I

Conducting a Program, Project, or Process?” is intended to guide users through a series of questions to help determine if you are designing and implementing a program, project, or a process and should make use of the additional resources (project managers are guided to Appendix A and processes are guided to Appendix B). The graphic map shows which sections of the Toolkit are applicable to programs, projects, and processes.

Performance Management Toolkit

Section 1: Align Programs to Advance Existing Strategies

By the end of this section, the reader will understand how to align programs to advance existing strategies.

SECTION 1: ALIGN PROGRAMS TO ADVANCE EXISTING STRATEGIES

Programs can emerge in various ways. For example, some programs can be a change to existing efforts, some can be new efforts in a region or sector, and some may come with general parameters set forth by a policy or speech. When initiating program design and before doing more detailed planning, the first step is to review the program idea and assess how it can best align with and advance existing strategies or other high level directives. Program designers should consider which specific strategic goals and objectives the program could ultimately help achieve, and then design it in a way to maximize its contribution towards higher level strategy.

As you begin to design the program, project, or process it is important to take the time to consider the bigger picture and to see how the future program can best advance existing strategies. Aligning programs to existing strategies will help operationalize strategic documents and support bureaus, offices, or posts in meeting goals and objectives. Aligning your intended program to existing strategies can help reduce the risk of lost time or resources as you further your strategic goals and objectives. Some important strategies and reports to consider include:

Joint Regional Strategy (JRS)

Functional Bureau Strategy (FBS)

Integrated Country Strategy (ICS)

USAID Country Development Cooperation Strategies (CDCS)

Sustainable Development Goals (SDGs)

Interagency policies, sector strategies, Presidential directives, commitments and guidance (e.g. National Security Strategy)

An office, bureau, or post should consider the following questions before designing a program to support the alignment and advancement of strategic plans. These questions can be used as considerations throughout program design and implementation to support continued alignment to strategies:

How can the program idea help achieve one or more of the FBS or JRS objectives or sub-objectives?

How does the program support the achievement of the relevant ICS(s) objectives?

How does the program concept help achieve any goals and objectives of agencies or groups outside of the State Department?

Is your program idea within the purview of your office, bureau, or post?

Case Study:

The President of the United States recently committed $5 million for Mary’s regional bureau, South Bureau, to address the US Government priority of global climate change. Mary is leading the design and management of the program that will fulfill this commitment. It is up to Mary and her team to determine how best to use these funds to address climate change; they will use this program design and performance management toolkit to guide the process.

Mary started the program design process by reviewing existing strategies to determine how the commitment will best advance them. She reviewed her bureau’s JRS and the President’s Climate Action Plan. In the JRS she identified the following goal and objective:

JRS Goal: Decrease greenhouse gases (GHG) emissions and other climate-warming emissions through Clean Energy programs.

JRS Objective 1: Build the capacity of institutions to implement low emission energy development strategies.

The President’s Climate Action Plan committed the United States to leading international efforts to combat global climate change and specifically addresses expanding clean energy use and reducing GHG emissions.

Mary also reviewed the ICSs of the countries in the region to determine in which country(s) to implement the program. Mary and her team determined that Freedonia’s ICS objectives and sub-objectives most closely align with and will be advanced by a climate change initiative. She also reviewed the relevant FBS and confirmed that clean energy is one of its goals.

Mary will move on to Section 2 of the Toolkit: Conduct a Situational Analysis, to fully understand the context within which she will be developing the program.

CASE EXAMPLE:

Align Program to Advance Existing Strategies

Considerations List Answers

How does the program idea help achieve the goals or objectives from the relevant JRS or FBS?

The program is focused on climate change and the goals and objectives of the JRS involve reducing GHG emissions.

How does the program help achieve one or more ICS objectives or sub-objectives?

The sub-objectives of Freedonia’s ICS align with reducing GHG emissions thereby this program can be designed to advance these existing strategies.

Does the scope of the program idea fit within the purview of your office, bureau, or post?

Yes, as it is in South Region. But Mary’s team will collaborate with the appropriate functional bureau.

How have you determined what is necessary for the program to achieve the goals and objectives of higher level strategy?

The program should be designed to focus on reducing GHG emissions in some capacity because climate change is a large sector for the program to advance existing strategies.

How does the program concept help achieve any goals and objectives of agencies or groups outside of the State Department?

The program will also advance the President’s Climate Action Plan committed the United States to leading international efforts to combat global climate change and specifically addressed expanding clean energy use and reducing GHG emissions.

How has the program implemented a plan to ensure continued alignment and advancement of strategy throughout program design and implementation?

The program will review the JRS, ICS, FBS, and President’s Climate Action Plan when the program design is completed after each year of implementation, and when these documents are updated.

Table 1. Case Example, Align Programs to Advance Existing Strategies

Section 2: Conduct a Situational Analysis

By the end of this section, the reader understand how to conduct internal assessments, external assessments, root cause analyses, and reviews of issues.

SECTION 2: CONDUCT A SITUATIONAL ANALYSIS

After assessing how best to align with the goals and objectives of existing strategies, the next step is to conduct a situational analysis. A situational analysis is a comprehensive review of the current state or conditions surrounding the program idea that could affect its design, implementation, or outcome.

It will help determine if your program design approach is feasible and increase your general awareness of the program environment.

Conducting a situational analysis allows you to understand the past and present contextual factors, why these factors exist, and if any steps need to be taken to account for them. The information gleaned from the situational analysis will inform your program problem statement, goals and objectives, and logic model later on in the program design process and may help you establish baselines for performance indicators developed during the performance management section (see Section 4).

STEP 2.1: WAYS TO CONDUCT A SITUATIONAL ANALYSIS:

The extent of your situational analysis will depend on the amount of time and resources available to you while planning your program. This section explains three possible ways to conduct the analysis:

document review, inter- and intra-agency coordination, and other external stakeholder coordination.

1. Document Review Purpose: Leverage existing information that can be collected independently and at minimal cost.

Documents to review may include but are not limited to:

JRS/FBS

ICS

USAID Country Development

Cooperation Strategy (CDCS)

Evaluations on similar programs or activities that have occurred in the region Secondary data (data that has already been collected by sources such as the World Bank, UN, country government statistics, etc.)

Program monitoring documents and performance metrics from similar programs or activities

2. Inter and Intra-Agency Coordination

Purpose: Prevent duplication of efforts and increase the scope of your program’s knowledge base by leveraging State Department and U.S. government partners.

Partners can include:

Subject matter specialists and program officers in your office, bureau, or post

Colleagues at post and desk officers in regional bureaus

Subject matter experts in the functional or regional bureau implementing similar programs

Subject matter experts in cooperating agencies (USAID, DOD, etc.)

3. Coordinate with External Stakeholders

Purpose: Collect outside perspectives on your program environment.

Coordinating with stakeholders involves reaching out to interested parties, such as:

Program beneficiaries

Local civil society organizations (CSOs) or non- governmental organizations (NGOs)

Local leaders

• International NGOs working in the sector or geographic area

• International donors

• Private sector

• Host Country Government officials

STEP 2.2: TYPES OF SITUATIONAL ANALYSES:

Not every program must undergo the same level of analysis depending on its nature and complexity, but key pieces of information should be obtained to understand if the program approach is possible and necessary. Using the three methodologies in Step 2.1, there are four types of assessments and reviews that should be conducted in your situational analysis:

2.2.1 Internal Assessment reviews what is happening within your own office, bureau, or post.

2.2.2 External Assessment reviews contextual factors that could affect your issue or problem.

2.2.3 Root Cause(s) Analysis looks at causes that are not immediately observable.

2.2.4 Review of the Issue or Problem examines the concern the program is designed to address.

2.2.1: Conduct an Internal Assessment An internal assessment1 allows you to understand the capabilities within your own office, bureau, or post. It is important to understand the financial, time, and human capital resources of your office, bureau, or post prior to designing a program to validate it is within your capabilities and capacity.

Important factors to consider include:

Program budget

Number of personnel available

Amount of time each person has available to dedicate to the program, and available dates

Workforce skills required to design and manage the program

Availability of skills within your office, bureau, or post

Method to procure the desired skills, if unavailable in your office, bureau, or post and where they can be found

Additional training required to design and manage the program, if applicable

Necessary Information Technology (IT) systems are in place within your organization to manage the program, if applicable

Mary started to design the program by conducting an internal assessment of South Bureau. Her internal assessment was designed to identify the resources and skills her team possesses and to be aware of any potential gaps prior to designing the program. Using the Internal Assessment Tool in Table 2, Mary asked a series of questions to bureau leadership and staff and found that the necessary resources are available to implement the program idea.

1 Conducting an Organizational Situation Analysis http://www.thesustainablengo.org/general-management-skills/conducting-an-organizational-situation-analysis

Internal Assessment

Questions Answers

What is the budget of the program? $5 million

How long are people needed? 2 years

How many personnel are available to work on the program? 3 people in the bureau

How much time does each person have available to support this program and during what periods?

1 program manager with 50% availability in the next two years who could manage overall

2 staff who could add support as needed over the next two years.

What specific skills and knowledge are required to manage this program?

Climate change subject matter expertise

Program management

Monitoring and evaluation (M&E)

Training facilitation

Diplomacy and negotiation

Are these skills available in the office, bureau, or post? Bureau has climate change subject matter expertise and program management experience, but no M&E experience

If these skills are not available in your office, bureau, or post where will you find them?

Will contract M&E support

Will coordinate with Embassy for technical support in the field

Is additional training required to design and manage the program?

No

Is the current information technology platform sufficient to operate the program?

The post in Freedonia has the infrastructure to support program implementation including necessary IT (e.g., computers, internet, phones, etc.)

Table 2. Case Example, Internal Assessment

2.2.2: Conduct an External Assessment

An external assessment is the process of surveying the environmental surroundings where the program will take place to establish baseline context and understand potential opportunities and threats.

Certain portions of the external assessment may not apply to domestic programs, to conduct an external assessment use Table 3. Resources to use for an external assessment include but are not limited to:

National or international regularly published surveys and indices

Government statistics

Secondary data (e.g., vulnerability analysis, analytical reports from other agencies, etc.)

Coordination with external stakeholders (e.g., interviews, focus groups, surveys, etc.)

ICS and country-specific strategies from other US and international agencies

F Country Data Analytics packages, F Interagency Network Databank (FIND), or other available data analytics

Factors to consider include but are not limited to:

1. Political/Legal Factors: These are the government actions at any level (e.g., federal, local, municipal, district-wide, regional, national, etc.) that could affect a program, including any law(s) that prohibit or inhibit similar programs.

2. Security Factors: These are the factors that involve information security considerations for processing classified program information, or threat to personnel associated with the program (i.e.

State Department employees, implementers, or the stakeholders/recipients of the program). This is a wide range of considerations that may include but are not limited to war or instability in the region, high or critical technical threat location, violent crime rate (e.g., assault, robbery, kidnapping, hate crimes, etc.).

3. Environmental Factors: These are the natural surroundings that may influence the implementation of the planned program including, but not limited to:

– Geography and the cost considerations related to implementing in the planned geographic location (e.g., mountainous, desert, landlocked, rural vs. urban, etc.)

– Health factors (e.g., infectious disease that may inhibit implementation of the program)

– Natural disaster factors (e.g., flooding, hurricane, blizzard, drought, etc.)

– Climate (i.e., seasonal considerations)

These factors may influence the size or scope of the program, cost of the program, or time of year the program can be implemented. The environment may be a major obstacle to implementing the program, but it could also be the very reason the program exists; for example it may be a response to an economic crisis or to a natural disaster.

4. Cultural Factors: These are the societal norms and customs that will influence the implementation of the program. This includes but is not limited to considerations for religion, gender, LGBT issues, ethnicity, language, general cultural expectations, etc.

5. Economic Factors: These are the external economic factors that should be considered when implementing this program. These include, but are not limited to:

http://find.state.gov/

– Costs of operating in the location

– Poverty levels in the location

– Economic differences among ethnicities

– Economic differences among gender

– Factors influencing economic disparities/economic development

– Economic factors influencing participation

6. Institutional Factors: This includes an analysis of specific organizations with whom you may be working and how the program, project, or process may complement or compete with other programs and activities. This may include the processes, attitudes, customs, and behaviors of organizations that may influence the implementation of the program.

7. Infrastructure Factors: This includes the physical infrastructure that the program will be operating in and the access to necessary amenities such as water, power, electricity, physical office space and technology (e.g., cell phones, computers, internet, etc.).

Mary conducted an External Assessment to learn about the environment in which the program would operate, and any key contextual factors that could affect later design steps. She considered each of the factors listed above in her assessment and included the information in Table 3.

External Assessment

Questions

Description

Answer

Could this negatively interfere with program implementation?

Political/Legal P o l i t i c a l L e g a l L e g a l v

Are there political/legal factors that could positively or negatively affect operations or activities?

Laws, regulations, hostility towards such activity

Government already pledged to reduce GHG emissions by 25% by 2030 through clean energy initiatives.

Weak policy and regulatory environment

Low tariffs on petroleum and coal imports

□ Yes

No

□ Maybe, but not critical

□ N/A

Notes or mitigation actions, if necessary:

Security P o l i t i c a l L e g a l L e g a

Are there security factors to consider?

Instability, violent crime

The region is stable and peaceful, however the country is known for crimes such as robbery, theft towards expats and wealthy residents resulting in restricted movement at night and in certain sections of urban areas.

□ Yes

□ No

Maybe, but not critical

□ N/A

Notes or mitigation actions, if necessary:

Personnel travelling to the country must take precautions when travelling at night and in particular areas, and should read State Department’s travel information.

Environment P l i t i c a l L e g a l L e g a

What are the natural surroundings that may influence the implementation of the planned program?

Geography, health, natural disaster, climate, etc.

The country has an ocean border and experiences occasional hurricanes.

Malaria is prevalent in the country.

The country has a diverse topography with mountains, hills, plains, and highlands.

The country has extensive river systems.

□ Yes

No

□ Maybe, but not critical

□ N/A

Notes or mitigation actions, if necessary:

P o l i t i c a l L e g a l L e g a

Culture P l i t i c a l L e g a l L e g a

What are the cultural factors that may influence the implementation of the program?

Consider the societal norms and customs that will influence the implementation of the program.

General population and economic sector is heavily reliant on coal sector.

□ Yes

No

□ Maybe, but not critical

Economics P l i t i c a l L e g a l L e g a

What are the economic factors that may influence the implementation of the program?

Consider the factors external to your office, bureau, or post such as the economy of the country in which the program will be implemented, economic well-being of the customers/ recipients, the costs of operating, etc.

Clean energy sector is extremely underdeveloped. In addition to a policy framework, the country needs private investment to support new clean energy infrastructure, operations, and maintenance.

Per-capita income is low and many existing businesses do not have spare capital so costs will need to be comparable to the cost of existing energy sources.

Coal sector is highly integrated into economic fabric, so there could be resistance to new energy sources.

People are unaware of business opportunities clean energy development could bring to region.

□ Yes

No

□ Maybe, but not critical

Table 3. Case Example, External Assessment

Institutional

What are the institutional factors that may influence the implementation of the program?

Consider the characteristics of any organizations you may need to partner with such as: customs, attitudes, processes, behaviors, etc.

The World Bank implements a large power infrastructure program in the region called En-R-Ge, which is a potential partner.

The World Bank has a history of partnering with the Department.

The World Bank has funding available to support infrastructure that would complement the work of Mary’s program.

□ Yes

No

□ Maybe, but not critical

2.2.3: Determine the Root Cause(s)

A Root Cause Analysis is a method used to take a closer look at the source of the problem(s) or issue(s) your program will address. Often, the symptoms of a problem are observable, but the problem itself is not. In order to effectively address the true problem, one must engage in deeper analysis to uncover its root cause(s).

One way to do a root cause analysis is through the Five Whys Approach as seen in Table 4. The Five Whys Approach is a technique where you ask a series of “why” questions starting with an undesirable symptom you would like to address. From there, ask why that particular situation is happening.

Continue asking why each situation is happening until you reach what you believe to be the original cause of the undesirable symptom. You may not find it necessary to complete all five “Whys” or may find it necessary to ask more questions.

Mary and her team used the Five Whys Tool to understand the root cause of Freedonia’s high rates of GHG emissions. Understanding the root cause will allow Mary’s team to design a program that addresses the correct issues. They started by asking, “Why are there high rates of GHG emissions?”

and kept asking why. Table 4 shows how Mary’s team used the Five Whys Tool.

Five Whys

Component Description Answer

What is the undesirable symptom you would like to address?

The visible result of the underlying root cause. The problem we would like to solve.

The country has high rates of GHG emissions.

First Why:

Why is this happening?

Why does the country have high rates of GHG emissions?

The country has a high dependence on coal-burning energy.

Second Why:

Why does the country have high dependence on coal energy?

The country lacks the infrastructure for clean energy.

Third Why:

Why does the country lack the infrastructure for clean energy?

The government does not have the funding for large-scale infrastructure projects and lacks the necessary frameworks, regulations, and policies to implement a clean energy program.

The private sector and general population do not demand clean energy.

Fourth Why:

Why does the government lack the necessary frameworks, regulations, and policies?

Why does the private sector and population not demand clean energy?

Historically, the country’s energy has come from coal; its policies and infrastructure reflect this. Most people in Freedonia are more concerned with the cost of energy than its source.

Fifth Why:

Why don’t the people of Freedonia care where their energy comes from?

The people of Freedonia are unconcerned or unaware of the harmful health effects of burning coal or the economic possibilities new businesses could bring to the area.

Team Discussion

Does the answer to the fifth why reveal a root cause to address?

Yes. Increasing the capacity of Freedonia’s government is critical to implementing a clean energy program and decreasing GHG emissions. It is also critical to raise awareness in the private sector and the general population about the health and economic benefits of clean energy.

Table 4. Case Example, Five Whys

2.2.4: Conduct a Review of the Issue

The review of the issue is a close examination of the problem you would like to solve that takes into account the larger context of the external assessment and a greater understanding of the root causes. This step allows for deeper understanding of stakeholders and/or beneficiaries and what efforts are occurring in the geographic area related to this problem.

Information includes but is not limited to:

1. Beneficiaries or customers of the program

After learning the root causes and perpetuating factors of the problem, you can understand who the beneficiaries or customers of your program should be. Asking questions such as:

– Who is the relevant audience for the program? (e.g., group, age, gender, ethnicity, location, etc.)

– What are their needs?

– Why do those needs exist? (referring to external assessment and root cause analysis)

– How can their needs be met considering the root cause(s) and available resources?

2. The systems, culture, and behaviors that keep the problem in existence

Using information from the external assessment, consider the systems, behaviors, culture, beliefs, power structures, economics, security, and other factors that perpetuate an issue or problem.

3. Other actors and potential partners and relevant stakeholders

– It is important to ask if there are other organizations (federal agencies, NGOs, CSOs, local leaders, private sector partners, etc.) working in this area. Knowing this information can help you prevent a duplication of efforts, discover potential partners with whom you can work, and how you can learn from the other programs.

– Stakeholders include anyone who may be affected during or after the program implementation.

4. Unmet need for existing programs

In the process of identifying other factors, it is beneficial to assess the strengths and weakness of the programs in place. This can help you learn what programs have been successful and why.

This analysis can help identify where there are gaps in current and past programs and where your program can fill a void.

Mary conducted a document review and coordinated with interagency stakeholders to review the issue of GHG emissions in Freedonia. This information is important as it will inform the development of the problem statement, the goal(s) and objectives, the program logic, etc. The information from this analysis can be found in Table 5.

Review of the Issue or Problem

Questions Description Answers

What are the root causes of the problem?

The underlying systemic causes of the problem inserted from Step 2.2.

Lack of infrastructure

Lack of capacity to develop frameworks, regulations, and policies needed to implement a clean energy program

Lack of funds

Lack of awareness of health and economic benefits of clean energy

What are the systems and behaviors that keep the problem in existence?

The systems, behaviors, culture, beliefs, power structures, economics, instability, etc. that perpetuate an issue/problem.

Existing frameworks, regulations, and policies do not promote clean energy and reinforce coal as a primary energy source.

Reliance on coal for commercial and residential energy needs

Who are the beneficiaries/ customers of the program?

Who will benefit most from the program? Who is in greatest need?

Freedonia Government

People of Freedonia

Businesses in Freedonia

What are the needs of those recipients?

Consider the population you would like to target. What are their greatest needs?

Improved infrastructure

New frameworks, regulations, and policies

Funding and investment

Decreased health risks from poor air quality

How can these needs be met?

Taking into account the root causes of the problem, immediate and long-term needs, how can the program address the needs of the recipients?

Support the building of hydro and wind power infrastructure by:

– Building capacity and providing technical assistance to create sound frameworks, regulations, and policies and generate funding and investments;

– Raising public awareness of the benefits of clean energy.

Who are the other actors or potential partners working in the area?

Consider other actors in the area and sector who may be implementing similar programs.

During the external assessment Mary and her team learned that the World Bank was implemented the En- R-Ge program, a power infrastructure program in Red and Blue Districts that focused on clean energy sources.

Table 5. Review of the Issue or Problem

Other Recommended Analyses for Consideration:

1. Risk Analysis: It is important to consider risk throughout the entire program design and performance management process. Please refer to the 2 FAM 30 for more information on risk assessment and risk management.

2. SWOT Analysis: Another useful tool to consider when conducting a situational analysis is the SWOT analysis tool. A SWOT analysis identifies strengths, weaknesses, opportunities, and threats of implementing a program or project.

Other Recommended Analyses for Foreign Assistance Programs:

3. Gender Analysis: As stated in USAID ADS 205, a gender analysis is important as it ascertains the:

a. Differences in the status of women and men and their differential access to assets, resources, opportunities and services;

b. Influence of gender roles and norms on the division of time between paid employment, unpaid work (including subsistence production and care for family members), and volunteer activities;

c. Influence of gender roles and norms on leadership roles and decision-making; constraints, opportunities, and entry points for narrowing gender gaps and empowering females; and

d. Potential differential impacts of development policies and programs on males and females, including unintended or negative consequences.

When conducting a gender analysis refer to USAID ADS 205 for more guidance in conducting your gender analysis.

4. Country Tropical Forest and Biodiversity Analysis: As detailed in USAID ADS 201 the incorporation of this country-level analysis into the design and implementation process can enhance the resiliency of over-exploited natural resources, improve environmental health, and strengthen partner-country environmental governance essential to achieving goals and objectives. Refer to USAID ADS 201 for more information on conducting this analysis.

5. Sustainability Analysis: Sustainability issues should be considered during program design by conducting a Sustainability Analysis as part of your situational analysis. As stated USAID ADS 201, under most circumstances, program outcomes are expected to be sustainable, meaning https://www.usaid.gov/sites/default/files/documents/1870/205.pdf https://www.usaid.gov/sites/default/files/documents/1870/205.pdf https://www.usaid.gov/sites/default/files/documents/1870/201.pdf https://www.usaid.gov/sites/default/files/documents/1870/201.pdf https://www.usaid.gov/sites/default/files/documents/1870/201.pdf https://www.usaid.gov/sites/default/files/documents/1870/201.pdf they continue or evolve under their own momentum or actions, without continued donor intervention. Posts are asked to analyze key sustainability issues and considerations around a host of issues including:

– Economic

– Financial

– Social soundness

– Cultural

– Institutional capacity

– Political economy

– Technical/sectoral

– Environmental

Section 3: Design a Program

By the end of this section, the reader understand how to develop problem statements, program goals and objectives, logic models and theories of change, and program summaries.

SECTION 3: DESIGN A PROGRAM

STEP 3.1: DEVELOP THE PROBLEM OR NEEDS STATEMENT

The first step in Section 3 is to develop a problem or needs statement. This is a clear description of the issue, challenge, or need the program seeks to address and serves as the program focus. Use the information from Section 2 to form a comprehensive problem statement that is aligned with your office’s, bureau’s, or post’s goals and objectives. Problem statements should take into account the scope, context, location, demographics, and stakeholders related to the program.

Developing a problem or needs statement provides the basis and reasons for implementing a program.

The problem or needs statement condenses the information in Section 2: Conduct a Situational Analysis – External Assessment (to understand the environmental context), Root Cause Analysis (to understand underlying causes of the problem) and Review of the Issue (to understand and narrow in on specifics surrounding the issue) – to form a clear, succinct, detailed description of the problem.

To develop a problem statement, answer: who, what, where, when, why, and how:

1. Identify Who

– Understand the demographics of the possible beneficiaries or customers

– Understand the key stakeholders engaged in or who may be affected by the program’s activities

2. Identify What

– Understand the scope and subject matter of the issue

– Understand the extent, or the size and scale of the issue

– Understand the severity, or the gravity, and urgency of the issue

3. Identify Where

– Describe the geographic or specific locations where the issue takes place

– Understand the location of the issue or problem

4. Identify When

– Understand the timing of the issue or problem, if applicable

– Understand contextual factors that affect when it occurs

5. Identify Why

– Understand the cultural impact surrounding the issue or problem: politics, beliefs, customs, norms, power structures, etc.

– Understand the underlying root causes of the issue or problem

– Understand perpetuating factors that give rise to the issue or problem

6. Identify How

– Review the customers and stakeholders identified in “who” and discuss more specifically how this issue affects this population

– Review gender analysis (if conducted)

7. Develop the Problem or Needs Statement

The previous six answers combine to help form the problem statement, which should be a clear description of the issue or problem the program would like to address. Walking through the problem statement in terms of who, what, where, when, why, and how enables you to understand the context around the issue you would like to solve. A case example is provided in Table 6 below.

Mary and her team used information from the situational analysis to answer specific questions about the problem: who is affected, to what extent the problem affects them, where the issue occurs, when, why and how. Mary and her team used the Problem or Needs Statement Tool to determine the exact challenge their program will address. This information will be used in the development of the goals and objectives and lays the foundation for designing the program. Please see the information from Mary’s team in Table 6.

Develop the Problem or Needs Statement

Question Category Definition Answer

WHO Stakeholders and/or

Beneficiaries

Describe who is involved in the issue or challenge

Government officials at the Ministry of Interior Bureau of Energy lack capacity to create a clean energy program.

Private manufacturing sector does not want to invest in clean energy.

World Bank En-R-Ge Program has committed to building wind and hydro power systems in Red and Blue districts.

General population lacks awareness of climate change and the benefits of clean energy.

WHAT Scope, Extent, Severity

Describe the:

Sector, area, or subject matter

Scale or magnitude

Gravity and urgency

Sector: Clean Energy – based on geography and previous studies, wind and hydro are the best forms of clean energy for Freedonia

Scale: implement clean energy programs in Blue and Red districts

Urgency: Freedonia pledged to decrease GHG emissions by the year 2030 by 25%

WHEN Context Describe when the issue or challenge takes place. Be sure to include any specific context or circumstances under which the issue or challenge occurs such as time of day, time of year, or time in one’s life.

At the national level the need is persistent, and becoming more urgent as the time for meeting the GHG reduction commitment approaches. At the local level, the need for clean energy is most acute, as rates of respiratory illnesses are increasing.

Question Category Definition Answer

WHERE Location Describe the geographic or specific location where the issue occurs.

Districts of Red and Blue report the greatest dependency on coal energy for both industrial and personal use.

Red and Blue districts have the geography to support wind and hydro energy.

World Bank program in these districts

WHY Causes Describe the root causes of the issue or challenge.

Lack of infrastructure

Lack of capacity to develop frameworks, regulations, and policies needed to implement a clean energy program

Lack of funds

Lack of awareness of health and economic benefits of clean energy

HOW Effect How does the issue affect the customer or beneficiary?

Burning coal has polluted the air and led to a high incidence of respiratory illness.

The pollution has made it unfeasible to develop a tourist industry.

There are no clean energy jobs in Freedonia.

Problem

Statement

Using the information described in the six categories above, write a detailed problem statement to guide the program design process.

Current GHG emission levels should be curbed or reduced to improve long-term health and economic outcomes in Freedonia, but the government lacks the capacity to create and implement the policy and regulatory frameworks necessary to develop a Clean Energy sector and citizens are not aware of the benefits of clean energy sources.

Table 6. Develop the Problem or Needs Statement

STEP 3.2: DEVELOP PROGRAM GOALS AND OBJECTIVES

The terms goals and objectives are often used interchangeably, as both point toward the anticipated results of the program, but each serves a different purpose. Goals are broad and indicate overarching aims that the program contributes to, but may not necessarily be achieved within the timeframe of the program. Objectives are narrower, more specific, and should be realized within the program timeframe. Program objectives should articulate desired results that will help accomplish program goals. There are usually two to five program objectives per program goal.

Note that goals and objectives can be created separately, or as part of your logic model exercise described in Step

3.3 of the Toolkit.

Program Goal(s) explain the overall intent or purpose of the program to which program objectives and subsequent activities are expected to contribute. Goal(s) focus on the desired outcomes. Program goal(s) should be:

Broad and visionary: Program goals are ambitious in nature but also realistic. They should be specific enough to clearly communicate program aims, express future direction and vision by using active or directional verbs, and avoid overly technical terms or acronyms.

Long-term: Program goals may extend beyond the timeframe of the program.

Not program resource-dependent: Program goals should not be limited by program resources because they represent a broader desired end state to which the program is contributing.

Strategy alignment: Program goals should align with and advance existing strategies.

Objectives are the highest level result the program can affect or achieve towards accomplishment of the program goal(s). They are statements of the condition(s) or state(s) the program is expected to achieve within the timeframe and resources of the program. To enable sound performance management, objectives should ideally incorporate SMART principles: Specific, Measureable, Achievable, Relevant, and Time-bound.

Specific: Describes the desired result in concrete and clear terms such that anyone reading it should interpret it in the same way

Measurable: Can be evaluated and/or assessed against some standard such that it is possible to know when the objective is met

Achievable: Attainable within allotted time and resources

Relevant: Linked to achieving the program goals

Time-Bound: If applicable, describes when the objective will be complete and any check points relevant to program goals, activities, or resources

Mary and her team used information from the situational analysis to develop the problem statement: Current GHG emission levels should be curbed or reduced to improve long-term health and economic outcomes in Freedonia, but the government lacks the capacity to create and implement the policy and regulatory frameworks necessary to develop a Clean Energy sector and citizens are not aware of the benefits of cleaner energy sources. This information will be used in the development of the goals and objectives in Figure 1. Upon developing the goals and objectives, Mary and her team will develop the Program Logic Model.

Develop Program Goals and Objectives

Problem Statement: Current GHG emission levels should be curbed or reduced to improve long-term health and economic outcomes in Freedonia, but the government lacks the capacity to create and implement the policy and regulatory frameworks necessary to develop a clean energy sector and citizens are not aware of the benefits of cleaner energy sources.

Program Goal: Increase the supply of and demand for clean forms of energy as a means to decrease GHG emissions and improve health and economic outcomes.

Objective 1: Increase capacity of the national government to create and implement regulatory frameworks

Objective 2: Increase public awareness of and demand for clean energy

Figure 1. Case Example, Program Goals and Objectives

STEP 3.3: DEVELOP A PROGRAM LOGIC MODEL AND THEORY OF CHANGE A logic model is a tool to systematically document and visually represent program investments, activities, desired results, and the relationship between them. The process of creating a logic model is one of the most critical steps in both program design as well as performance management. It allows the program team and key stakeholders to work through how and why they think the program will work to achieve established goals and objectives and it becomes the basis for designing monitoring and evaluation plans.

Establishing what the program is expected to achieve sets the foundation for what to monitor and evaluate progress against. This section describes each logic model component and how to work through its completion. A theory of change is a brief statement that ties your logic model together by summarizing why, based on available evidence and consideration of other possible paths, and the changes described in your logic model that are expected to occur.

One of the key benefits of creating a logic model is that it gets all program stakeholders on the same page in terms of how and why the program will be implemented. When creating a logic model, it is therefore critical that all key stakeholders are involved or consulted, including program implementers, managers, key decision-makers, monitoring and evaluation staff, and possibly even intended program beneficiaries or customers.

The components of a logic model include:

Short- and Long-Term Outcomes – what we achieve

Outputs – what we get

Activities – what we do

Inputs – what we invest

Using the diagram in Figure 2 as a reference point, each component from left to right is intended to build on or result from the previous one. Outcomes are the intermediate and long-term results of the outputs.

Outputs are the results of the activities and activities are based on the planned program inputs. When initially creating a logic model, however, it is most helpful to start from right to left whereby stakeholders first discuss and agree upon the intended outcomes so the other components lead to these desired long-term results.

Figure 2. Completing a Logic Model

3.3.1: Develop Long-Term Outcomes

Long-term outcomes describe system- or societal-level results, and can focus on behavior, normative, and policy changes the program seeks to achieve. Though often not realized for years after program implementation, long-term outcomes are based on what success would look like if the program goal(s) were ultimately achieved. Figure 3 provides an example of a strong long-term outcome. Figure

4 below provides an illustration of how the program goal(s) can be used to develop the long-term outcomes section of the logic model.

Long-Term Outcome

Reduction in incidence in air-quality related disease.

This is a strong long-term outcome because although it will not be realized during the timeframe of a single climate change program, it is a logical long- term extension of a climate change program and it is a measurable/observable change at the broader societal level.

Figure 3. Case Example, Long-Term Outcome

Developing Long-Term Outcomes

Program Goal: Increase the supply of and demand for clean forms of energy as a means to decrease GHG emissions and improve health and economic outcomes.

Long-Term Outcomes: Decrease in GHG emission, decrease in the number of respiratory illnesses

Figure 4. Developing Long-Term Outcomes

3.3.2: Develop Short-Term Outcomes

Short-term outcomes describe the immediate effects of the program and often focus on changes to the knowledge and attitudes of the program’s beneficiaries or customers, such as a change in attitude as a result of an awareness campaign or a change in behavior due to a training program. The short-term outcomes of the program should be linked to the program objectives. Figure 5 provides an example of a strong short-term outcome. Figure 6 provides an illustration of how the program objectives can be used to develop the outcomes section of the logic model.

Short-Term Outcome

New clean energy policies, plans, regulations and frameworks implemented.

This is a strong short-term outcome because this is the result of the capacity building program and indicates a…