Attachment_3_-_USAID_Mozambique_Portfolio_Review_Mission_Order.pdf

MOZAMBIQUE

MISSION ORDER

S U BJ EC T : Portfolio Reviews ORDER NO . M0203-6

F I L I N G : Series 200 - Programming Policy CHRON NO .

DATE ISSUED: DAT E EFFECTIVE :

Upon Signature

SUPERSEDES : NIA

AU TH 0 RI TY : See Section 11 below PAGE 1 OF 9 PAGES

I. PURPOSE

The purpose of this Mission Order is to provide procedural guidance and establish roles and responsibilities for Mission Portfolio Implementation Reviews {PIR).

II. AUTHORITY/POLICY

• ADS 203.3.12 regarding portfolio reviews.

• ADS 203.3.13 regarding Program Cycle learning.

Ill. DEFINITIONS

• Adaptive Management (ADS 201.3.12): Regardless of the approach to analysis, the analytic basis for projects should be continually updated, tested, and upgraded. Project design should incorporate plans to reflect on the evidence underlying project design and assess the implications of divergence between anticipated and unanticipated outcomes during project implementation. If necessary, a Mission should undertake additional analytic work and make necessary adjustments to the project design as the project is implemented. A Mission that has included a focus or component in its CDCS or Project on collaborating, learning, and adapting should have a separate implementation plan for operationalizing this component across the Mission portfolio. It should ensure that project designs reflect the projects' relationships to the learning implementation plan.

• Critical Assumptions (ADS 200.6): A general condition under which the development hypothesis, or strategy for achieving a CDCS Development Objective, will hold true. Critical assumptions are outside the control or influence of USAID and its partners (in other words, they are not results), U.S. Agerv;;y for International Development JATComplex Rua 1231 , n.0 41 P.O.Box783 Maputo, Mozambique

Tel: (258) 21 35 20 00 Fax (258) 21 35 21 00 www.usaid.g<:Nlmz.

U.SA. Postal Address 2330 Maputo Place U.S. Department d State Washngton, D.C. 20521-2330 but they reflect conditions likely to affect the achievement a Results Framework or project logical framework, such as political stability, the level of world prices or the openness of export markets.

• Development Hypothesis (Theory of Change) (ADS 200.6): Describes the theory of change, logic, and causal relationships between the building blocks needed to achieve a long-term result. The development hypothesis is based on development theory, practice, literature, and experience, is country-specific, and explains why and how the proposed investments from USAID and others collectively lead to achieving the Development Objectives (DOs) and ultimately the CDCS Goal. It is a short narrativ,e that explains the relationships between each layer of results (in the Results Framework - see section 3 below), upwards from the sub-Intermediate Results (sub-IRs), to the IRs, the DOs, and the CDCS Goal, often through if-then statements that reference the evidence that supports the causal linkages.

• Game Changers: Emergent and/or dynamic broad conditions that are beyond the Mission's control, are evolving in ways that are difficult to anticipate with much certainty, and could impede or facilitate strategy implementation. Monitoring of these conditions should be done based on associated tripwires that may trigger programmatic and project contingencies or even changes in strategic direction. The status of "game changers," if any, will be covered in the strategic- level PIR.

• Management Action Tracker: An Excel table outlining management actions and decisions taken during the PIRs. The table should include details such as the decision/action taken, date, deadline, responsible office(s), and status. The Management Action Tracker will be stored in the Mission's Pll at P:\Pll\PR0\07_MnE\08_Management Action Tracker.

These issues should be revisited periodically for the sake of accountability; not only at the subsequent PIR, but as part of agenda for other meetings where the Mission sees this as feasible (e.g., normally scheduled one-on-one meetings between office and Front Office, Quarterly Financial Reviews (see Budget MO), Project Reviews (see Monitoring MO), etc.)

• Portfolio Alignment (ADS 201 .3.4.3): During CDCS development, the Mission should identify how the results expected to be achieved by its current portfolio of activities align with the new RF. Based on an analysis of which results from its current portfolio are expected to be incorporated into the new RF, either without modification or with some modification, the Mission begins initial planning for the alignment of activities described in

201.3.12.3. During project design, the Mission should validate the findings of the portfolio alignment review (201 .3.8) with respect to those activities which are proposed to be continued under the project with or without modification. Project Design Teams should integrate ongoing activities into Concept Papers and Project Appraisal Documents (PADs) if they contribute to the Project's Purpose or will be modified to do so. During project design, the Mission should define how the on-going activities should be structured, managed, and staffed to best implement the project.

- 2 -

• Program Cycle Learning (ADS 203.3.13): Throughout the program cycle, learning is fundamental to an adaptive approach to development. While learning is not new, Missions can consider using a strategic learning and adapting plan to maximize development results (specific methods, including some examples of Mission practice with a Collaborating, Learning and Adapting Approach can be found at: http://kdid.org/kdid lab/library). This plan can help the mission and implementing partners coordinate their efforts, collaborate for synergies, learn more quickly, and make iterative, timely course corrections. Learning encompasses a systematic and deliberate approach to:

o Generate, capture, share, analyze and apply information and knowledge, including performance monitoring data as well as findings from evaluations, research, practice, and experience;

o Engage with local thought leaders and development actors to complement context indicators with deep contextual knowledge and experience;

o Coordinate efforts within the mission and among partners and other stakeholders to increase synergies;

o Facilitate collaborative learning and extend the mission's influence and impact beyond its project funding;

o Combine critical analysis and periodic reflection with adaptive management processes and agile funding mechanisms to maintain relevance as new learning emerges and/or the broader context or country conditions change.

• Strategic Management: This phrase is related to, but conceptually broader than, more familiar terms such as "Results-Based Management," "Result-Oriented Management," or "Managing for Results." Strategic Management subsumes change and knowledge management and encompasses strategic planning, but goes beyond it, as well. Associated principles include:

o Focus on results o Learn from the past o Look ahead and anticipate o Be proactive, not reactive o Collaborate and leverage other efforts o Value knowledge and its effective management o Actively manage change, both planned and unanticipated

A strategic manager embraces learning and will periodically ask him/herself, ''What needs to change? What has changed? What do I know? What do I need to know? Whom do I need to influence? What opportunities can I exploit?"

IV. ROLES AND RESPONSIBILITIES

All USAID/Mozambique offices are responsible for attending PIRs and understanding the principles and procedures contained here. The Program Office (PO) shall lead the Mission PIR process on behalf of the Mission Director. The PO is also responsible for updating this Mission Order (MO) as a

-3-result of further guidance from USAID/Washington or from learning gained from application of the MO. Development Objective (DO) Teams are responsible for ensuring that activity/ implementing mechanisms (IMs) and project-level reviews are held during the year (see Performance Monitoring MO) and for allocating sufficient time to adequately analyze results and prepare for the PIR.

V. PROCEDURES

A. Overview and Policy

The Program Cycle graphic (See Annex A) includes a central circle labeled "Learning and Adapting." The explicit emphasis on learning is a key feature of current USAID programming policy that distinguishes it from past policies. Focusing on learning will transform the way PIRs are conducted. More reflection, more questions, and more evidence will be needed. All participants will need to be open to the notion that "we got it wrong" or "that assumption did not hold true" or "it is not working like we thought it would" and work together, re-examining evidence to propose course corrections and make decisions. Making program changes based on evidence, especially for alternative scenarios, will not be an admission of failure, but a mark of distinction for the heads-up manager. Learning itself will need to be valued in the review: the review should not only ask, "how are we doing?" but also, 'What did we learn?", "What should we change?", 'What the data/evidence tells us?", and "What works and doesn't?".

ADS 203.3.12 requires that Missions conduct at least one PIR per year focused on the higher levels of the Country Development Cooperation Strategy (CDCS) Results Framework, but recommends that Missions programming $20 million or more per year conduct two reviews. It is USAID/Mozambique policy that two PIRs will be conducted per year with somewhat different purposes and procedures behind each. The PIR conducted at the end of the fiscal year, generally around November or December, will be focused on the higher levels of the results framework.

The mid-year PIR, generally conducted around May or June, will focus on illustrative issues further discussed below and may vary depending on the needs of the Mission. Timing of both PIRs must be considered along with similar Pl Rs of the Mission's Presidential Initiatives, such as the President's Emergency Plan for AIDS Relief (PEPFAR), Feed the Future, the President's Malaria Initiative, and any others.

The Strategic PIR that focuses on the higher levels of the results framework should analyze at minimum: DO performance; a set of indicators covering all levels (DO, Intermediate Result (IR), and sub-IR) from the Mission's Performance Management Plan (PMP); as well as relevant evaluation findings and action plans based on those findings.

ADS 203.3.12 lists questions of issues that should be considered in the higher level review; these illustrative PIR content and discussion issues are repeated here for reference:

-4 -

• Status of critical assumptions and the Development Hypothesis defined in the Results Framework, along with the related implications for performance

• Country and regional trends and how the context is evolving

• Evidence that projects are leading to the achievement of the DO (the cause and effect relation)

• Status of cross-cutting themes and/or synergies between DOs

• Status of related partner efforts that contribute to achievement of IRs and DOs

• What has been learned during project implementation from monitoring data, evaluations, partners, or other sources of evidence

• Procurement Plan discussion and pipeline analysis

The discussion themes and subjects for the second Portfolio Review, again, conducted generally in May or June, will vary depending on the particular needs of the Mission. The following illustrative examples should be considered for this mid-year review:

• ProjecUoperational focus, including Activity M&E plans, project team effectiveness and staffing, etc. (see ADS 203.3.12 for lists of issues to examine in such a review)

• Focus on particular indicator data: progress of indicators in relation to targets and quality and feasibility of data

• Focus on cross-cutting implementation issues

• In-depth focus on a few key projects within the overall program context

• Discuss conducting a similar PIR with host country counterparts

• Focus on results and impacts in relation to host country national policies and priorities

• Procurement plan discussion and pipeline analysis

• Environmental Compliance (required): the Mission will review the status of environmental compliance for each activity/IM, per the requirement in ADS 204.3.3. Environmental sustainability is integral to USAID's overall goal, and therefore must be mainstreamed into all activities to achieve optimal results, to avoid inadvertent harm to the people we are trying to help, and to prevent wasting taxpayer dollars.

To meet this goal, USAID incorporates environmental considerations into results-based planning, achieving, and assessing and learning.

The COR/AOR/AM must ensure that their 22 CFR 216 environmental reviews are accomplished in a timely fashion so as not to unnecessarily delay implementation of any activities. The PO will work with the Environmental Officer and each COR/AOR/AM to create an Environmental Compliance Chart (see Annex B) that will be included in the PIR materials (PIR Binder). Any significant issues will be addressed during the PIR presentation.

B. Process and Procedures

PIRs follow a process consisting of three general steps. These steps will apply to the Strategic PIR but may be modified depending on the nature of the second PIR:

-5-

• Pre-Portfolio Review Stage: Planning and preparation undertaken by both the Mission PO and DO Teams; discussion of data cleaning and analysis; creating PIR agenda; and agreeing on PIR materials

• Portfolio Review: The review itself, referred to as the "PIR Presentation," where all Mission staff are invited to attend and participate. The DO Team will present its material (as determined by the guidance from PO), and staff will have the opportunity to discuss issues and interact in a Mission-wide review forum.

• Portfolio Review Follow-Up: After the PIR Presentation, the PO and DO Teams will ensure that issues discussed in the PIR are addressed and resolved, resulting in improved program/activity performance. Such follow-up actions ensure that the PIR exercise is a useful strategic management tool.

Step 1: The Pre-Portfolio Review Stage (approximately 6 weeks)

• Distribute documentation and guidance: Approximately 6 weeks before the review, the PO will provide the DO T earns with the official minutes and the "PIR Follow-up Action Memo" from the previous PIR, along with the Management Action Tracker maintained by the PO (see Definitions and Annex C). Each DO Team will use the minutes, the Follow-up Action Memo, and the Management Action Tracker to begin preparing for the next review. The PO will work with each office to schedule the PIR Presentation and provide overall guidance, including preparatory steps, required PIR materials, and special guidance resulting from consultations with the Front Office.

• Data collection and cleaning: The PO's Performance Monitoring point of contact (PMPOC) (see Monitoring MO) will work with DO Teams and other M&E Specialists to run reports from the Mission's performance monitoring information system (see Monitoring MO).

DO Team Leaders, project managers, and AORs/CORs/AMs (responsible for different levels of performance monitoring) will review the data quality and gaps. The PO will provide any needed common formats to facilitate data analysis and comparison. It is general practice that the COR/AOR/AM for each implementing mechanism be required to complete the PIR Activity Sheet (see Annex D); its usage will be determined for each individual PIR.

• Data analysis: It is critical that prior to the review, the DO Team and others decide what is expected from the data analysis, and understand what the data mean and possible directions for adaptive management: does performance monitoring data show everything to be working as expected? Does it signal poor implementing partner (IP) performance or something deeper? Does it reveal an unexpected positive trend that the team should take advantage of?

It is important that these meetings be held at higher levels- Project or DO, rather than activity/IM level-to understand the underlying patterns of change within the existing context. It may be helpful to involve external analysts (from another DO team, PO, Embassy) to

-6-help find patterns and objectively assess what the data mean. The DO T earns and PO will also need to decide how the data can be used to raise and respond to strategic questions and issues.

• Per the Budget MO, financial data and performance (e.g., pipelines, burn rates, additional funding needs, budget reallocations) will be reviewed on a quarterly basis through Quarterly Financial Reviews (QFR). The QFR will be conducted during this pre-PIR stage because it serves as a discussion point for each DO's PIR. Any issue raised during the QFR will be discussed, if relevant, in the PIR.

• Pre-Portfolio Review Meetings: DO Teams will conduct pre meetings internally with their own DO and with other Offices and DO T earns prior to the PIR. The nature of these meetings will vary depending on the particular review and the issues needing attention.

Pre-meetings may cover:

o More intensive data analysis by project teams to understand patterns (geographic, by IR, sub-IR, or by IP) o Meeting with host country counterparts and/or IPs to discuss issues o Meeting of all IPs involved in a project or even DO to discuss partners' views on progress, challenges, lessons learned, analytic gaps, collaboration/coordination needs, and potential course corrections.

o Pre-meetings may be Project Reviews or Activity Reviews (see Monitoring MO for more detail), that could prepare the team(s) for the Portfolio Review.

• Portfolio Review Agenda Preparation: The PIR timing will determine the main structure and function of the PIR. Strategic-level issues listed in ADS 203.3.12 will inform the agenda preparation.

The DO Team will work with the PO to finalize its PIR Presentation agenda. In general, the agenda will include the DO Team's PIR presentation; discussion issues; results of QFR meetings and pipeline analyses, programmatic planning, including procurement planning; evaluation and learning activities; and a review of action items from the previous PIR.

• Preparation and Review of Materials: The expanded DO T earn is responsible for preparing review materials as explained in the guidance distributed by the PO. Below is an illustrative list of the materials that should be submitted to the PO approximately ten working days prior to the scheduled review. The final list of materials to be included in the PIR binder will be confirmed when the PO issues the PIR Guidance.

o Issues Paper (see below) o DO Results Framework o Activity Sheets o Evaluation Report (Evaluation Plan, summary evaluations conducted over the reporting period) o Financial and Pipeline Analysis o Procurement Plan

-7-

Once the DO Team submits the PIR Binder to the PO, the PO will conduct its own review and provide any comments, questions, or request for clarifications back to the DO Team. If necessary, the DO Team will provide revised materials back to the PO for finalization.

Once finalized, the PO will circulate the PIR Binder to the Front Office and to the Mission.

• PIR Discussion Issues List: Additionally, when the PIR Binder is submitted to the PO for review, the PO will review the materials and provide a summary of "PIR Discussion Issues Lisr that will be given to the Front Office along with the final version of the PIR Binder.

This PIR Discussion Issues List will help guide discussion on the most urgent issues facing the DO. Potential decision points might be indicated (change in management structure, resource allocation, activity/IM modifications, etc.). The PIR Discussion Issues List will be distributed to the DO team before the PIR Presentation, with sufficient time to digest the material and be ready to answer the issues at the Review.

Step 2: The Portfolio Implementation Review

• The PIR Presentation will vary in time length based on the size of the

DO's portfolio. However, it is recommended that a PIR Presentation is no less than two hours. The Mission Director or his/her designee will chair all PIR Presentations.

• The DO's PDO backstop will be responsible for ensuring that minutes are taken at each PIR Presentation. The minutes will record highlights presented in at the PIR Presentation, notable successes, issues raised, action items, and decisions made. This report-out could also address was learned from the PIR. Action items will be compiled into a "PIR Follow-Up Action Memo" and copied into the Management Action Tracker.

Step 3: Portfolio Review Follow-Up (1-12 months) For the PIR process to be an effective strategic management tool and means to foster learning within the Mission, the discussion must be well documented and the decisions and other results consistently followed up and acted upon.

• PDOs will finalize the PIR Presentation minutes and the PIR Follow up Action Memo, by clarifying any ambiguous points and gaining clearance per Delegation of Authorities MO. PIR minutes will be stored in the Mission's Program Information Index (Pit) at P:\PJJ\PR0\07 _ MnE\05 _PIR\03 _Jssues_Papers.

• PDOs will maintain the Management Action Tracker (see Definitions) by updating the status of old items reviewed during the PIR and adding new items that resulted from the latest PIR. Action items should not be left for the next PIR, but are to be followed up periodically to ensure progress. This Management Action Tracker will also assist the DO Teams in ensuring that issues are followed up and resolved.

-8-

• The PO will organize at least one PIR Follow-up Meeting. This meeting will discuss the impact of review: what, if any, information or learning emerged from the PIR that could support effective programming in other DOs, among IPs, or among other counterparts.

Participants should develop a plan for sharing PIR findings, discussion and decisions, as appropriate.

VI. EFFECTIVE DATE AND CANCELLATION

This Mission Order is effective on the date of signature and will remain in force until cancelled or amended.

Date Jh (;1-/ )

-9-

Annex A: Program Cycle Graphic

Annex B: Environment Compliance Chart

DO 2 Portfolio Review

June 2, 2014

DO#:

Activity Implementing IEE Filed Env. IEE Propsoed Categorical Type of Additional Additional Partner (Yes=Date) Concerns Mitigation Exclusion? Environmental Environmental Mitigation Measures from IEE Measures (Yes=Date) Compliance Issues Needed (note if n/a)

Annex C: Management Action Tracker Template

Management Action Tracker Template (MATT) (To use, fill in te.xt in ll"· en lughhght. Delete explanatory comments in blue lughlig~r

MATTfor XXX

PDO Responsible for Overseeing MA TI:

[name, Office, Ext]

A new MATI will be created after each PR for each DO.

DO Team/Tee: me of DO

I

*Decision/ action taken Deadline du, (Describe the decision/action as discussed in the PR

Specify the deadline .

Timing/frequency of these actions f add rows for additional decisions/actions}

Responsible office(s)

Who is responsible for implementing the decision/action taken?

Status

Who is responsible for tracking the decision/action should specify here the how is the implementation of the decision/action?

[* These issues should be revisited periodically for the sake of accountability; not only at the subsequent PR, but as part of agenda for other meetings where the Mission sees this as feasible (e.g., normally-scheduled one-on-one meetings between office and Front Office}

Annex D: Portfolio Review Activity Sheet Template

ED Activity Sheet: FY 13 Q 1-Q2 [ACTIVITY NAME]

General Information

Activity Name:

Implementing Partner(s) (including sub-partners):

COR/AOR/AM Name:

Proiect Start and End Date: (for "field support"/centrally managed activities, list date support begins/ends)

Geographic Focus: (provinces and districts):

Objectives: (from the contract/grant - "what we want to achieve over life of the project"):

Expected Results for FY 13: (from the annual workplan - "results we expected to achieve this year"):

Financial Information

FinanciaJ Information Funding Funding Comments Source A Source B

I. Funding Source (DA, etc.)

2. Field Support (central or Bilateral)

3. Life-of-Project Funding

4. Cumulative Obligations to Date

5. Mortgage (#3 - #4) 6a. Disbursement: FY 20 I I 6b. Disbursement: FY 2012 6c. Expenditures: FY 2013 (Ql+Q2) 6d. Expenditures: FY2013 (projected for the year) 6e. Expenditures: Proportion of FY 2013 Expenditures on TA

vs. Commodities

7. Total Expenditures (since project start)

8. Average Monthly Expenditure (since project start) (#7 I # of months active)

9. Pipeline (#4 - #7) I 0. Pipeline Months (#9 I #8)

Indicator FYl2 FYl3 FY13QI- Comments Actual Target Q2

("On target," "Underperforming," or Actual "Overperforming," fo llowed by brief explanation, if necessary)

Activity Narrative Over Past Six Months (3,000 character limit)

Major Implementation Issues/Constraints and Recommended Actions

Implementation Issues/Constraints Recommended Actions

Upcoming Major Actions/Events Over Next Six Months

Past Six Months' Performance Management Actions

Actions Date Comments