MCC-16-RFQ-0015_0001_-_Final.pdf

5. PROJECT NO.

CODE 7. ADMINISTERED BY

2. AMENDMENT/MODIFICATION NO.

CODE

6. ISSUED BY

8. NAME AND ADDRESS OF CONTRACTOR

4. REQUISITION/PURCHASE REQ. NO. 3. EFFECTIVE DATE

9A. AMENDMENT OF SOLICITATION NO.

9B. DATED

PAGE OF PAGES

10A. MODIFICATION OF CONTRACT/ORDER NO.

10B. DATED

1. CONTRACT ID CODE

FACILITY CODE CODE

Offers must acknowledge receipt of this amendment prior to the hour and date specified in the solicitation or as amended, by one of the following methods:

The above numbered solicitation is amended as set forth in Item 14. The hour and date specified for receipt of Offers

E. IMPORTANT:

is extended,

(a) By completing Items 8 and 15, and returning __________ copies of the amendment; (b) By acknowledging receipt of this amendment on each copy of the offer submitted; or (c) By separate letter or telegram which includes a reference to the solicitation and amendment numbers. FAILURE OF YOUR AC- KNOWLEDGMENT TO BE RECEIVED AT THE PLACE DESIGNATED FOR THE RECEIPT OF OFFERS PRIOR TO THE HOUR AND DATE SPECIFIED MAY is not extended.

12. ACCOUNTING AND APPROPRIATION DATA

(REV. 10-83)

is required to sign this document and return ___________ copies to the issuing office. is not, A. THIS CHANGE ORDER IS ISSUED PURSUANT TO: (Specify authority) THE CHANGES SET FORTH IN ITEM 14 ARE MADE IN THE CONTRACT ORDER NO. IN ITEM 10A.

15C. DATE SIGNED

B. THE ABOVE NUMBERED CONTRACT/ORDER IS MODIFIED TO REFLECT THE ADMINISTRATIVE CHANGES SET FORTH IN ITEM 14, PURSUANT TO THE AUTHORITY OF FAR 43.103(b).

RESULT IN REJECTION OF YOUR OFFER. If by virtue of this amendment you desire to change an offer already submitted, such change may be made by telegram or letter, provided each telegram or letter makes reference to the solicitation and this amendment, and is received prior to the opening hour and date specified.

C. THIS SUPPLEMENTAL AGREEMENT IS ENTERED INTO PURSUANT TO AUTHORITY OF:

D. OTHER

BY

Contractor

16C. DATE SIGNED

14. DESCRIPTION OF AMENDMENT/MODIFICATION

16B. UNITED STATES OF AMERICA

Except as provided herein, all terms and conditions of the document referenced in Item 9A or 10A, as heretofore changed, remains unchanged and in full force and effect.

15A. NAME AND TITLE OF SIGNER 16A. NAME AND TITLE OF CONTRACTING OFFICER

15B. CONTRACTOR/OFFEROR

STANDARD FORM 30 NSN 7540-01-152-8070

PREVIOUS EDITION NOT USABLE Prescribed by GSA - FAR (48 CFR) 53.243

(Type or print) (Type or print)

(Organized by UCF section headings, including solicitation/contract subject matter where feasible.)

(No., street, county, State and ZIP Code)

(If other than Item 6)

(Specify type of modification and authority)

(such as changes in paying office, appropriation date, etc.)

(If required)

(If applicable)

(SEE ITEM 11)

(SEE ITEM 13)

(X)

(X)

13. THIS ITEM APPLIES ONLY TO MODIFICATIONS OF CONTRACTS/ORDERS,

IT MODIFIES THE CONTRACT/ORDER NO. AS DESCRIBED IN ITEM 14.

11. THIS ITEM ONLY APPLIES TO AMENDMENTS OF SOLICITATIONS

AMENDMENT OF SOLICITATION/MODIFICATION OF CONTRACT

(Signature of person authorized to sign) (Signature of Contracting Officer)

1 7

11/20/2015

Millennium Challenge Corporation Contracts and Grants Management Division 875 15th St, NW Suite 200 Washington DC 20005-2227

Millennium Challenge Corporation Contracts and Grants Management Division 875 15th St., NW, Suite 200 Washington, DC 20005-2227 Ray Staniewski, Contract Specialist (KTR) (202) 521-3656

To all Offerors/Bidders

MCC-16-RFQ-0015

11-10-2015

X

X X

** HOUR & DATE for Receipt of Offers is EXTENDED to: 1:00 PM (EST) 12/07/2015

The purpose of Amendment 0001 to Solicitation MCC-16-RFQ-0015 is to:

a) Extend the response date and time from 1:00 PM (EST) 11/30/2015 to a new due date & time of 1:00 PM (EST) 12/07/2015.

b) Post all questions received with answers provided by MCC.

c) Amend SECTION J to add Annex XII – Project Evaluability Assessment (Draft)

d) No other changes are made to this Amendment.

OFFERORS SHALL ACKNOWLEDGE RECEIPT OF THIS AMENDMENT BY INCLUDING A SIGNED COPY IN VOLUME II OF THERE QUOTATION.

staniewskirj Highlight staniewskirj Highlight staniewskirj Highlight staniewskirj Highlight staniewskirj Highlight

MCC-16-RFQ-0015 – AMENDMENT 0001 Page 2 of 7

CONTINUATION PAGE

BLOCK 14. DESCRIPTION OF AMENDMENT/MODIFICATION (Continued)

Solicitation #MCC-16-RFQ-0015

Impact Evaluation of MCC’s Senegal Irrigation and Water Resource Management Project

(1) Given that the questions are being submitted on 17 November and MCC would probably not be able to answer all those received before 20 November, this likely leaves offerors with only three working days to produce their proposals before the Thanksgiving holiday. We ask that MCC extend the deadline to at least December 4 or 10 working days after the responses to questions are issued to allow sufficient time to fully incorporate MCC’s responses into our quotation.

RESPONSE: MCC will extend the response deadline through December 7, 2015; a longer extension is not possible given the broader deadlines surrounding this project. Offerors are hereby put on notice that the initial mission to Senegal will need to occur very soon after contract award (early January 2016, at the latest).

Offerors shall incorporate this timeframe into their Illustrative Work Plan (see Section L.7.8.4) as confirmation that their proposed team will be available for the initial mission.

(2) Section B.5 states that a firm-fixed price (FFP) award is planned for the base period and that the Option Periods are intended to be Time & Materials (T&M) in nature (with the option for negotiating a conversion to FFP at a later date prior to the exercising of each Option Period. Given (a) the degree of uncertainty about the details of the design that might be selected by MCC for implementation in Option Periods I and II (if MCC decides to approve the proposed evaluation design and proceed with the evaluation and (b) that the RFQ indicates that negotiation of the budget for the Option Years will take place after MCC assesses the Base-Year design and work plan, please provide guidance on how the option periods should be priced in this initial quotation so that all offerors can respond under a consistent set of assumptions. For example: Should Offerors assume the existing evaluation design remains valid and would be completed as originally planned? Or can MCC provide illustrative allocations of level of effort (LOE) and a plug figure for data collection costs (as MCC has done in other similar RFQs—most recently for the Cape Verde LMI evaluation RFQ)?

RESPONSE: The Base Period must be priced on a FFP basis in accordance with the format at Section L.8.6.1 of the solicitation; the option periods must be priced on a T&M basis in accordance with the formats prescribed at Sections L.8.6.2 and L.8.6.3, with the understanding that MCC may negotiate for these option periods to be converted to FFP prior to exercise.

As for assumptions, Offerors shall develop their budget based on the current design (as reflected in Annex I), while taking into account the known changes to the design that are noted in Sections C.3.1 through C.3.5.

Should the final approved design vary substantially from the current design as outlined in the solicitation, MCC will take that into account prior to exercising the next option period/project phase.

(3) Related to the FFP nature of the Base Period, please clarify if the offeror is required to provide proposed payment amounts for the schedule of payments shown in Section F.2.3 or if these will be negotiated later as part of the award process. (If a proposed schedule is to be provided as part of the quotation, please confirm that the payment amounts are only required for the Base Period, as the Option Periods are priced on a T&M basis.)

RESPONSE: Offerors must use the price templates provided at Sections L.8.6.1 through L.8.6.3 of the solicitation for Phases I, II, and III, respectively. The price template for Phase I (Base Period) mirrors the schedule of payments shown in Section F.2.3 for Phase I. The Offeror is expected to fill in their proposed FFP in the “Associated Costs” column for each deliverable listed in the table, with the exception of the

MCC-16-RFQ-0015 – AMENDMENT 0001 Page 3 of 7 deliverables with a grey box in that column, which indicates that there will be no payment associated with that deliverable. The sum of the amounts in each of the six white boxes will be the Offeror’s proposed FFP for Phase I.

Indeed, Option Periods 1 and 2 (which correspond with Project Phases II and III, respectively) should be proposed on a T&M basis in accordance with the price templates at Sections L.8.6.2 and L.8.6.3. The schedule of payments shown at Section F.2.3 for Phases II and III will be used if/when a FFP is negotiated for those phases after contract award and prior to option exercise, per the footnote in the price schedule at Section B.1 of the solicitation. The schedule of payments shown at Section F.2.3 for Phases II and III should not be filled out by the Offeror at this point in time.

(4) Section C.2 states “MCC seeks a new contractor… [emphasis added]”. Please clarify if the statement with regard to MCC seeking a new contractor is related to concerns with the previous evaluation design or its implementation (i.e., if there are already known issues/problems with the existing design (such as comparison-group contamination and/or evaluability)) or is a new contractor required for administrative reasons (i.e., the previous contractor is no longer available to/accessible by MCC).

RESPONSE: MCC seeks a competitive bidding process for this evaluation requirement. Though MCC cannot elaborate on the specific reasons for this new procurement, all of the currently known concerns about the evaluation design have been described in Section C. Further, although the described concerns do exist and MCC seeks an evaluator who is willing and able to critically assess the feasibility of the current design, MCC does not currently anticipate dramatic changes to the evaluation design.

(5) Concerning Section C.3.6.2, the subsequent subsections do not provide any detail on what is included in MCA “oversight” of the data collection firm. Please provide additional detail on what this data collection oversight entails so that we may appropriately budget for any coordination with the MCA that is required during data collection.

RESPONSE: All data collection efforts, including design, contracting, implementation, and oversight are the responsibility of the Offeror. The Senegal Compact ended on September 23, 2015 and the MCA as such will cease to exist after the end of the 120-day administrative closure period (January 20, 2016). The Government of Senegal has established the “Unité de Suivi” as a post-compact entity that will continue to provide support to Compact activities and to the related M&E activities. However, as it relates to the evaluation and corresponding data collection, the role of the post-Compact entity is that described in Section C.3.6.2. They will primarily provide feedback and input on evaluation deliverables. And they will, to the extent possible, provide support to the Evaluator in country (such as assistance in coordinating stakeholder meetings, coordinating stakeholder feedback on evaluation deliverables, etc.). Support from the post- Compact entity is not expected to extend to significant logistical or technical support nor oversight to the data collection efforts. Thus, Offerors should incorporate all costs and logistics for data collection within their quotes.

(6) Concerning Section C.4.5.3 please clarify what MCC means by “evaluation materials.” Does this mean just final deliverables, draft and final deliverables, or other materials as well? It would be helpful if MCC could list which “evaluation materials” need to be translated, so we may budget translation costs appropriately.

RESPONSE: “Evaluation Materials” refer to all:

Survey questionnaires Interview or Focus Group questionnaires/protocols Consent forms/protocols Surveyor/field worker manuals or guidebooks Training materials for data collection staff (surveyors, supervisors, etc.)

Any other data collection materials, tools, protocols, and supporting documentation

MCC-16-RFQ-0015 – AMENDMENT 0001 Page 4 of 7

For translation purposes, all data collection tools should be translated (survey questionnaires, interview/focus group questionnaires and protocols, consent forms and protocols, etc.). Survey/field worker manuals/guidebooks and training materials for data collection staff should be produced in the language most suitable for the context (most likely French), but do not need to be translated back to English.

(7) Please confirm that a completed copy of Sections B.1 and Section K of the RFQ is to be included in the cost quotation. (They are not listed as being required in Section L.8.)

RESPONSE: Yes, a completed copy of Section B.1 and Section K.1 shall be included in Volume II of the quotation. The following sections are hereby added to Section L.8 of the solicitation:

L.8.7 Offerors must include a completed copy of the table at Section B.1.

L.8.8 If an Offeror has already completed their annual representations and certifications in the System for Award Management (SAM), please include a paragraph substantially similar to paragraph (b)

(2) of the clause at Section K.1. Offerors that have not completed their annual representations and certifications in SAM must include the completed text of Section K.1 as part of Volume II of their quotation.

(8) L.7.8.2.1 would appear to be incomplete: “Discuss the Contractor’s relevant experience, particularly experience in designing and implementing impact evaluations using a randomized control trial methodology; and.” Was further text to be added to complete this requirement?

RESPONSE: Section L.7.8.2.1 is hereby modified to read as follows:

“Discuss the Contractor’s relevant experience, particularly experience in designing and implementing impact evaluations using quasi-experimental designs and managing the complexities of data collection and analysis therein.”

The previous wording was in error and is deleted in its entirety.

(9) Requirement L.7.8.2.1 isn’t directly related to requirement L.7.8.2. Is this intended to be presented under L.7.8.2 or as a separate section?

RESPONSE: Section L.7.8.2.1 was intended to be a separate section. However, the section number will remain unchanged, with the understanding that it is not directly related to Section L.7.8.2.

(10) Given that the work to be carried out in Option Periods I and II will be dependent on the design developed during the Base Period and thus is not known in detail at this point, we request that MCC revise requirement L.7.8.4 such that the illustrative work plan be provided for the Base Period only.

RESPONSE: As with the budget, Offerors shall design their quotation around the current methodology and the changes described in Section C. Offerors must develop their illustrative work plan based on these assumptions.

(11) Please clarify the following points in Section M.3.1.1: (a) With respect to the provision of reference information related to evaluation methods proposed, is MCC’s intent that offerors should, in the past performance table shown in Section L.7.9 include in the project description/relevance cells of the table details on the evaluation methods used? If so, please provide guidance on the level of detail that would satisfactorily address MCC’s requirement for this item. (b) The reference information requested appears to be limited to the infrastructure sector; if this is indeed MCC’s intent, we request that reference information form the agricultural sector also be considered, given many of the research questions seem more closely aligned with that sector than the infrastructure sector.

RESPONSE: With respect to part (a) of the question, yes, Offerors should, in the past performance table shown in Section L.7.9, include in the project description/relevance cells of the table details on the evaluation methods used. For instance, it would be helpful for Offerors to briefly indicate the relevant

MCC-16-RFQ-0015 – AMENDMENT 0001 Page 5 of 7 methodology for which they are providing a reference.

With respect to part (b), the bolded sentence at Section M.3.1.1 is hereby deleted and replaced with the following:

“The contractor shall submit specific reference information related to evaluation methods proposed and the irrigation and/or agriculture sectors.”

(12) The Evaluation Design Report raises the possibility of contamination of the control group with respect to the land tenure intervention on p. 23. In particular, the report mentions that some households in the control area were anticipated to receive a set of land-tenure related services similar to those that the MCC land tenure component would provide to the treatment households. Can MCC provide any information on the extent to which control households actually did receive these services?

RESPONSE: This contamination did occur. It affected approximately half of the geographic comparison area. Thus, approximately half of the geographic comparison areas received the Land Activity interventions.

An analysis has not yet been done regarding the actual number of surveyed households that are affected by this contamination, however.

(13) Since the Compact ended on 23 September 2015, were all activities, projects, and tasks listed in the Compact for the irrigation and land components implemented and completed in full, besides the Social Safe Guard Measures Activity? If not, what remained unimplemented?

RESPONSE: With the noted exception of the Social Safe Guard Measures Activity, all Project Activities and interventions were completed by the end of the Compact.

(14) What is the Government estimate for this solicitation per phase?

RESPONSE: MCC will not provide the Government estimate for this solicitation.

(15) Per Section H.5, what phases of the proposed work are funded incrementally?

RESPONSE: If Phases II and III remain T&M type, they will likely be incrementally funded. MCC’s intent is to fully fund any FFP phases, if possible.

(16) Can MCC please clarify expectations on Task 3 activities listed under Section C.4.4, with respect to the evaluation contractor’s responsibilities: "Monitor program implementation and conduct risk assessment" (also listed as a set of deliverables listed under Phase I under Section F.2.3). Other sections of the RFQ suggest that program implementation was completed in September 2015. If some program implementation activities are still being carried out, can MCC clarify which activities these are?

RESPONSE: Implementation of the Senegal Compact has indeed been completed. Thus, the role of the Evaluator in regards to Task 3 is that of assessing and verifying project implementation and identifying current and future risks to the validity of the evaluation. Section C.4.4 of the solicitation is hereby deleted and replaced as follows to clarify the role now that the Compact has ended:

C.4.4. Task 3: Monitor program implementation and conduct risk assessment C.4.4.1. The Contractor is responsible for assessing program implementation according to project and evaluation design and documenting any deviations from the design. This requires collaboration with MCC, MCA sector leads and stakeholders to ensure continuous support for and commitment to the evaluation design.

C.4.4.2. For Senegal, this requires an assessment report (which is separate from standard Quarterly Reports as described in Section F.1.) of implementation according to the treatment (to ensure the intervention is implemented in the agreed treatment areas) and control (to ensure the intervention is not implemented in the agreed control areas).

MCC-16-RFQ-0015 – AMENDMENT 0001 Page 6 of 7

C.4.4.3. The Contractor is responsible for flagging any serious concerns or risks regarding the internal validity of the evaluation.

(17) Section F.2.3 assumes payment by deliverable as shown in the deliverables table and is not consistent with T&M pricing for Phases II & III (Labor & ODCs, respectively) as well as Sections H.4(4) & (5), page

32. Please advise.

RESPONSE: This is a hybrid type contract, with Phase I being FFP and Phases II and III being T&M (with the option to negotiate a conversion to FFP upon option exercise). Indeed, the table at Section F.2.3 assumes FFP for all phases; Offerors are only expected to complete the Phase I portion, which is the only phase that is definitively FFP at this point in time. The price templates at Sections L.8.6.1 through L.8.6.3 shall be used to propose pricing for Phases I, II and III; these templates are consistent with the FFP nature of Phase I and the T&M nature of Phases II and III. Please refer to Question 3 for additional information.

Sections H.4(4) and (5) on page 32 include T&M type language to cover the possibility that Phases II and III remain T&M.

(18) Will discrete sets of contract clauses be established for the FFP vs. T&M Phases, as Section I.1 applies to T&M only?

RESPONSE: No, discrete sets of contract clauses will not be established. The clause at Section I.1, as written, does in fact apply only to T&M type contracts; the basic clause at FAR 52.212-4 – Contract Terms and Conditions – Commercial Items (May 2015) is hereby incorporated by reference at Section I.8 of the solicitation to cover the FFP phase(s) of this requirement.

(19) Section L.2.1 states that quotations should be provided in Word or PDF. Section L.5.6 states that they should be provided in Word and Excel. Please confirm that cost quotations may be provided in PDF.

RESPONSE: The second sentence at Section L.2.1 (“The most current format…”) is hereby deleted in its entirety and Section L.5.6 is amended to include PDF as an acceptable format for the narrative portion of the quotation. Cost quotations (i.e., the templates at Sections L.8.6.1 through L.8.6.3) must be submitted in Microsoft Excel format. Sections L.2.1 and L.5.6 are hereby deleted and replaced as follows:

L.2.1 All offers are to be submitted in electronic format. The offer due date is December 7, 2015 at 1:00 PM Eastern Standard Time to staniewskirj@mcc.gov.

L.5.6 In Microsoft Word or PDF for the narrative portions of Volumes 1 and 2, and in Microsoft Excel for the cost quotation required by Sections L.8.6.1 through L.8.6.3 (pricing for each phase should be separated by tab in the Excel spreadsheet).

(20) Per Section L.7.9, can we use Past Performance of subcontractors/consultants?

RESPONSE: At least two of the three required references must come from the prime contractor’s past performance; one of the three may come from a subcontractor/consultant. Also, Offerors may include additional references (over the required three) that may be related to a subcontractor’s/consultant’s past performance.

(21) Re: Section M. Evaluation, does MCC plan to assign scoring points to Technical Capability (Section M.2), Past Performance (M.3) and Price Evaluation (M.4)?

RESPONSE: Technical Capability (Factor 1) and Past Performance (Factor 2) will be given adjectival ratings. Price (Factor 3) will be evaluated in accordance with Section M.4.1.1; an adjectival rating will not be assigned to this factor. Numerical scoring will not be used.

(22) Could MCC please clarify if the matrices for the three current or past project examples be included in the Technical Quotation page count of 30 pages?

RESPONSE: Yes, the three (or more) past performance matrices required by Section L.7.9 must be mailto:staniewskirj@mcc.gov

MCC-16-RFQ-0015 – AMENDMENT 0001 Page 7 of 7 included within the 30-page limit of Volume 1 – Technical Quotation.

(23) Given the holiday weekend just before the submission deadline of November 30th, would MCC consider extending the deadline to allow for extra time to work on and provide more comprehensive proposals?

RESPONSE: See response to Question 1.

(24) Given the Thanksgiving holiday, could the MCC extend the proposal submission deadline to December 14?

RESPONSE: See response to Question 1.

(25) Could you please provide placeholder or “plug” figures for data collection costs and travel costs?

RESPONSE: MCC will not provide placeholder or “plug” figures for this requirement. See response to Question 2 for information on assumptions to inform your price quotation.

(26) Could MCC please provide LOE guidance for each phase of the evaluation?

RESPONSE: See response to Question 2 for information on assumptions to inform your price quotation.

(27) Task 3 in the base period (C.4.4) is to monitor program implementation and conduct risk assessment. Given that the Compact closed on September 23, 2015, what activities would be conducted under this task?

RESPONSE: See response to Question 16.

(28) Would you be able to provide the MCC Evaluability Assessment Tool referenced in the RFP?

RESPONSE: Yes, MCC can provide the current draft of the Evaluability Assessment Tool. It has been incorporated as Annex XII and is attached to this solicitation amendment.

(End of Q&A)

(End of Block 14)

Project Evaluability Assessment (PEA)

Background

As a results-based institution, MCC is dedicated to using rigorous methods to identify the core constraints to growth and design and implement evidence-based Projects in every Compact and Threshold country to address those core constraints.

For this reason, the concept of evaluability is central to MCC’s results agenda, as it is impossible to determine the results achieved by MCC projects unless they are designed up-front to be evaluable downstream. MCC defines evaluability as the ability of an intervention to demonstrate in measurable terms the results it intends to deliver. An evaluable intervention uses data to identify and verify the problem(s) it intends to address and underlying causes, has an evidence-based design to address the problem(s) identified, clearly identifies assumptions and risks associated with the intervention to address the problem(s), including identifying means for verifying and mitigating risks, and has clear and time-bound metrics for the results of the intervention in terms of outputs and outcomes.

Objective

The objective of an evaluability assessment is to use specific, transparent standards and best practices for assessing the following five dimensions of a project: (i) Problem Diagnostic, (ii) Project Objectives and

Logic, (ii) Risks and Assumptions, (iii) Project Participants/Beneficiaries, and (v) Accountability and

Learning Metrics.

Evaluability in Practice

A tool has been developed to help guide the iterative process of evaluability assessment.1 The Project

Evaluability Assessment tool draws on and expands on the standards and best practices identified in the

MCC Due Diligence Book in order to assess the five dimensions of a project through Compact development and into Compact implementation. The tool is intended to help guide teams to answer five big questions:

Is the problem clearly defined and is there sufficient evidence to support the problem diagnostic?

Are the project objectives and theory of change/logic clearly defined?

Are the risks and assumptions clearly defined with potential risk mitigation strategies?

Are project participants clearly defined and justified in terms of geographic scope and eligibility criteria?

Are the metrics for measuring results for both accountability and learning clearly defined?

In each of the five dimensions, the evaluability assessment tool lists a series of questions based on standards and best practices. While a Project may not meet all of the standards and best practices, and it is possible that the list of questions does not capture all potential factors that should be flagged, the purpose is to highlight the strengths and weaknesses of each intervention so that they can be addressed prior to Compact signature, in the development of the supplemental agreements, and during implementation as appropriate.

The tool may be applied throughout Compact development and implementation, and it is important to note that the assessment is intended to clarify and improve project documentation. For this reason, the assessment is only conducted on specific project documents as they are developed: (i) Concept Notes, (ii)

Concept Papers, (iii) Investment Memo, (iv) Other Project and Evaluation Design Document (s).

1 The evaluability assessment tool was developed through a yearlong pilot from 2012-2013, tested by the MCC M&E

Team on the following 7 Compact and Threshold programs: Threshold (Honduras); Implementation (Cape Verde II and Lesotho); Compact Development (El Salvador II, Georgia II, Ghana II, Benin II).

ANNEX XII

The current proposed process for assessment of project evaluability is intended to (i) maximize integration of the evaluability assessment within the Country Team, and (ii) ensure quasi-independent assessment for quality assurance. The proposed process is:

1. Concept Note: Country team members submit Project(s) Evaluability Memo to Country Team

Lead with a cc to respective MDs.

2. Concept Paper: Country team members submit Project(s) Evaluability Memo to Country Team

Lead with a cc to respective MDs.

3. Investment Memo: Country M&E lead works with Country Team to produce Investment Memo, reflective of current state of Project(s) Evaluability. Once submitted, an independent peer review of the IM, using the evaluability assessment tool, will be conducted. The composition and timing of this peer review is TBD.

At each stage, the intention is to provide feedback to the Country Team and Management on the current status of the Project(s) which is one of several inputs into project and evaluation selection, design, and implementation decisions.

Project2 – Evaluability Assessment

Assessment Categories Strengths, Weaknesses, and Next Steps

Dimension 1: Is the problem clearly defined and is there sufficient evidence to support the problem diagnostic?

Standards and best practices

1. Is there quantitative evidence regarding constraints to and sources of economic growth?

2. Is the problem(s) clearly defined and understood with sufficient evidence/quantitative (baseline) data available to support claims?

3. Is there evidence to support root causes identified?

4. Will all root causes be addressed by the proposed intervention or other, complementary intervention(s)?

5. Is there a public good rationale and/or market failure that necessitates government intervention and funding? This is linked to sustainability – how will recurrent costs be covered in the future, how will private sector investment be triggered, etc?

6. Is the institutional context understood, the political economy understood, and does the team clearly understand how the proposed intervention(s) link to other initiatives by the gov’t and/or other partners?

7. Is there a clear understanding on how different social and cultural dynamics (gender, poverty, race, ethnicity, etc.)

may be influenced by or influence the problem identified?

Strengths:

Weaknesses:

Next steps:

Dimension 2: Are the project objectives and theory of change/logic clearly defined?

Standards and best practices

1. Is the objective of the Project clearly stated with a link from the problem diagnostic?

2. Is the project logic and economic model clear, plausible and based on existing evidence and literature? If there is limited evidence, is there commitment to generating evidence via an impact evaluation (linked to Section 5)?

3. Is there a clear logic that links different projects within the Compact program that is then linked to accelerating economic growth?

4. Is it clear which component(s) of the problem diagnostic will be addressed by the proposed intervention, which will be addressed by complementary activities and which will remain risks for the MCC intervention to achieve proposed results?

5. Are the inputs, outputs, outcomes clearly defined and linked to the economic analysis (ERRs)?

6. Is the timeline for expected results clear and based on evidence?

2 It is anticipated that an evaluability assessment will be conducted for each Project; however this may vary depending on Compact composition. For example, the country team may prefer to conduct the evaluability assessment for individual Activities rather than Projects.

7. Is it clear whether or not benefits are expected to be sustained beyond the life of the compact?

Dimension 3: Are the risks and assumptions clearly defined with potential risk mitigation strategies?

Standards and best practices

1. Are the risks to achieving expected results clear, with clearly defined risk mitigation strategies?

2. Does the ERR reflect these assumptions and risks? Has sensitivity analysis been used to select key risks and assumptions?

3. Is it clear how risks will be monitored?

4. Is it clear how design and implementation may be altered as information on new risks/realization of risks occurs?

5. Does the project team make a critical assessment of the degree to which there may be blind spots or unknown unknowns in a project of this nature (e.g. how foreseeable are the potential risks that may arise in new sectors)?

Dimension 4: Are project participants clearly defined and justified in terms of geographic scope and eligibility criteria?

Standards and best practices

1. Is the selection criteria for project participants clearly defined and based on the problem and evidence in the program logic?

2. Is program participants’ selection based on credible, quantifiable selection criteria?

3. Are specific demographics (age, gender, poverty status) defined where necessary?

4. Are the geographic location(s) for the Project defined and based on the problem listed above and evidence in the program logic?

5. Will the Project design and implementation plan vary by different sub-groups and/or geographic locations based on the problem listed above and evidence in the program logic?

6. Can the selection be replicated for the purposes of an impact evaluation (linked with Section 5)?

Dimension 5: Are the metrics for measuring results for both accountability and learning clearly defined?

Standards and best practices

Are there clearly defined indicators and data sources identified for monitoring project implementation?

1. Are there clearly defined indicators for measuring expected performance (processes, outputs)?

2. Are the indicators linked to the ERR?

3. Is it clear which indicators will be disaggregated by gender, age, income as appropriate?

4. Is there a clear understanding of the time frame for expected results of each indicator (if varies)?

5. Is there sufficient information to set appropriate and feasible baseline and annual/quarterly targets?

6. Are there sufficient human and financial resources in the

MCA and IEs to conduct the necessary data collection/reporting during the life of the intervention?

Are data collection costs known and budgeted for?

7. Is it clear who will use the data and for what purpose(s)?

Next steps:

Are there clearly defined indicators and data sources identified for monitoring project results?

1. Are there clearly defined indicators for measuring expected performance (processes, outputs)?

2. Are the indicators linked to the ERR?

3. Is it clear which indicators will be disaggregated by gender, age, income as appropriate?

4. Is there a clear understanding of the time frame for expected results of each indicator (if varies)?

5. Is there sufficient information to set appropriate and feasible baseline and annual/quarterly targets?

6. Are there sufficient human and financial resources in the

MCA and IEs to conduct necessary data collection/reporting during the life of the intervention?

Are data collection costs known and budgeted for?

7. Is it clear who will use the data and for what purpose(s)?

Is the evaluation clearly defined for maximizing learning and accountability?

1. Is there commitment by all key stakeholders to implement the independent evaluation?

2. Are evaluation questions and outcomes clearly defined and prioritized?

3. Is it clear which outcomes will be disaggregated by gender, age, income as appropriate?

4. Is it clear who will use the evaluation results and for what purpose(s)?

5. Is the evaluation methodology the most rigorous and feasible possible?

6. Is it clear how an evaluation (performance or impact) will contribute to the evidence base in the sector?

7. Are there interim/continuous evaluation results which could help inform decisions during the compact life? If so, is such an evaluation built into the evaluation plan?

8. Do the potential benefits and learning from an evaluation of the program outweigh the costs?

9. Are there sufficient human and financial resources in the

MCC, MCA and IEs to conduct necessary data collection/reporting during the life of the evaluation? Are data collection costs known and budgeted for?