953324-19-Q-0011_-_2018-12-04_(Benin_II_Policy_Reform__Institutional_Strengthening_Evaluation).pdf

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL ITEMS

OFFEROR TO COMPLETE BLOCKS 12, 17, 23, 24, & 30

1. REQUISITION NUMBER PAGE 1 OF

2. CONTRACT NO. 3. AWARD/EFFECTIVE

DATE

4. ORDER NUMBER 5. SOLICITATION NUMBER 6. SOLICITATION ISSUE

DATE

7. FOR SOLICITATION

INFORMATION CALL:

a. NAME b. TELEPHONE NUMBER (No collect calls)

8. OFFER DUE DATE/

LOCAL TIME

9. ISSUED BY

13b. RATING

14. METHOD OF SOLICITATION

CODE

15. DELIVER TO 16. ADMINISTERED BY CODE

18a. PAYMENT WILL BE MADE BY CODE 17a. CONTRACTOR/

OFFEROR

CODE

FACILITY

CODE

CODE

TELEPHONE NO.

17b. CHECK IF REMITTANCE IS DIFFERENT AND PUT SUCH ADDRESS IN

OFFER

18b. SUBMIT INVOICES TO ADDRESS SHOWN IN BLOCK 18a UNLESS BLOCK

BELOW IS CHECKED

RFQ IFB RFP

SEE ADDENDUM

19.

ITEM NO.

20.

SCHEDULE OF SUPPLIES/SERVICES

21.

QUANTITY

22.

UNIT

23.

UNIT PRICE

24.

AMOUNT

(Use Reverse and/or Attach Additional Sheets as Necessary)

25. ACCOUNTING AND APPROPRIATION DATA 26. TOTAL AWARD AMOUNT (For Govt. Use Only)

28. CONTRACTOR IS REQUIRED TO SIGN THIS DOCUMENT AND RETURN

DELIVER ALL ITEMS SET FORTH OR OTHERWISE IDENTIFIED ABOVE AND ON ANY

ADDITIONAL SHEETS SUBJECT TO THE TERMS AND CONDITIONS SPECIFIED

29. AWARD OF CONTRACT: REF.

DATED . YOUR OFFER ON SOLICITATION

(BLOCK 5), INCLUDING ANY ADDITIONS OR CHANGES WHICH ARE

SET FORTH HEREIN, IS ACCEPTED AS TO ITEMS:

30a. SIGNATURE OF OFFEROR/CONTRACTOR

30b. NAME AND TITLE OF SIGNER (Type or print) 30c. DATE SIGNED

31a. UNITED STATES OF AMERICA (SIGNATURE OF CONTRACTING OFFICER)

31b. NAME OF CONTRACTING OFFICER (Type or print)

COPIES TO ISSUING OFFICE. CONTRACTOR AGREES TO FURNISH AND

OFFER

13a. THIS CONTRACT IS A

RATED ORDER UNDER

DPAS (15 CFR 700)

SEE SCHEDULE

12. DISCOUNT TERMS

ARE ARE NOT ATTACHED

ARE ARE NOT ATTACHED

27a. SOLICITATION INCORPORATES BY REFERENCE FAR 52.212-1, 52.212-4. FAR 52.212-3 AND 52.212-5 ARE ATTACHED. ADDENDA

27b. CONTRACT/PURCHASE ORDER INCORPORATES BY REFERENCE FAR 52.212-4. FAR 52.212-5 IS ATTACHED. ADDENDA

11. DELIVERY FOR FOB DESTINA-

TION UNLESS BLOCK IS

MARKED

10. THIS ACQUISITION IS UNRESTRICTED OR SET ASIDE: % FOR:

SMALL BUSINESS

HUBZONE SMALL

BUSINESS

SERVICE-DISABLED

VETERAN-OWNED

SMALL BUSINESS

WOMEN-OWNED SMALL

BUSINESS (WOSB)

ECONOMICALLY

DISADVANTAGED

WOMEN-OWNED SMALL

BUSINESS (EDWOSB)

8 (A)

NAICS:

SIZE STANDARD:

STANDARD FORM 1449 (REV. 3/2005) BACK

19.

ITEM NO.

20.

SCHEDULE OF SUPPLIES/SERVICES

21.

QUANTITY

22.

UNIT

23.

UNIT PRICE

24.

AMOUNT

32a. QUANTITY IN COLUMN 21 HAS BEEN

RECEIVED INSPECTED ACCEPTED, AND CONFORMS TO THE CONTRACT, EXCEPT AS NOTED:

41a. I CERTIFY THIS ACCOUNT IS CORRECT AND PROPER FOR PAYMENT

32b. SIGNATURE OF AUTHORIZED GOVERNMENT

REPRESENTATIVE

32c. DATE

41b. SIGNATURE AND TITLE OF CERTIFYING OFFICER 41c. DATE

42a. RECEIVED BY (Print)

42b. RECEIVED AT (Location)

42c. DATE REC'D (YY/MM/DD) 42d. TOTAL CONTAINERS

40. PAID BY

32d. PRINTED NAME AND TITLE OF AUTHORIZED GOVERNMENT

REPRESENTATIVE

32e. MAILING ADDRESS OF AUTHORIZED GOVERNMENT REPRESENTATIVE 32f. TELPHONE NUMBER OF AUTHORZED GOVERNMENT REPRESENTATIVE

32g. E-MAIL OF AUTHORIZED GOVERNMENT REPRESENTATIVE

33. SHIP NUMBER 34. VOUCHER NUMBER 35. AMOUNT VERIFIED

CORRECT FOR

PARTIAL FINAL

37. CHECK NUMBER

38. S/R ACCOUNT NO. 39. S/R VOUCHER NUMBER

36. PAYMENT

COMPLETE PARTIAL FINAL

95332419Q0011

TABLE OF CONTENTS:

Listing of Incorporated Purchase Requisitions Section B - Supplies or Services and Prices/Costs Section C - Description/Specifications/Work Statement Section D - Packaging and Marking Section E - Inspection and Acceptance Terms Section F - Delivery or Performance Section G - Contract Administration Data Section H - Special Contract Requirements Section I - Contract Clauses Section J - List of Documents, Exhibits, and other Attachments Section K - Representations, Certifications, and Other Statements of Offerors Section L - Instructions, Conditions and Notices to Offerors Section M - Evaluation Factors for Award

Listing of Incorporated Purchase Requisitions

Incorporated Purchase Requisition Numbers:

DPE-PR-19-0010

Section B - Supplies or Services and Prices/Costs

Item Number Base Item Number

Supplies/Services Quantity Unit

0001 Base Period: Benin II Policy Reform and Institutional

Strengthening

1 LT

Contract Type:Firm Fixed Price Unit Price

Extended Price

Description:

Period of Performance: Estimated 6 months

Est. PoP Start: 04/01/2019 Est. PoP End: 09/30/2019

Purchase Requisitions DPE-PR-19-0010

IDC Type: Not Applicable FSC Codes: R497 SUPPORT- PROFESSIONAL: PERSONAL SERVICES CONTRACTS

NAICS Code: 541990 All Other Professional, Scientific, and Technical S...

Supplies/Services Quantity Unit

1001 0001 Option Period I: Benin II Policy Reform and Institutional

Strengthening

Contract Type:Time and Materials Unit Price

Other Direct Costs Extended Price

Description:

Period of Performance: Estimated 17 months

Est. PoP Start: 10/01/2019 Est. PoP End: 02/28/2021

Purchase Requisitions

Option: 1 Option Time Date: 10/1/19 Option Time Duration: 516 Option Time Units: Days

Supplies/Services Quantity Unit

2001 0001 Option Period II: Benin II Policy Reform and Institutional

Strengthening

Contract Type:Time and Materials Unit Price

Other Direct Costs Extended Price

Description:

Period of Performance: Estimated 12 months

Est. PoP Start: 03/01/2021 Est. PoP End: 02/28/2022

Purchase Requisitions

Option: 2 Option Time Date: 3/1/21 Option Time Duration: 364 Option Time Units: Days

Item Number Base Item Number

Supplies/Services Quantity Unit

3001 0001 Option Period III: Benin II Policy Reform and Institutional

Strengthening

Contract Type:Time and Materials Unit Price

Other Direct Costs Extended Price

Description:

Period of Performance: Estimated 24 months

Est. PoP Start: 03/01/2022 Est. PoP End: 02/28/2024

Purchase Requisitions

Option: 3 Option Time Date: 3/1/22 Option Time Duration: 729 Option Time Units: Days

Clauses incorporated by reference

None

Clauses incorporated by full text

Section B

B.2. FAR 52.216-1 -- Type of Contract (APR 1984)

The Government contemplates award of a hybrid contract resulting from this solicitation. The base period will be Firm Fixed Price (FFP) while the option periods will be Time-and-Materials (T&M).

(End of Provision)

B.3. Funding Type

This contract will be funded, at time of award, with no-year Due Diligence funding. As such, funding obligated to this contract may be used during the life of the contract. The base period will be fully funded at the time of award and the option periods will be funded if/when the individual option periods are exercised.

B.4. Limitation of Funds – Incrementally Funded Contracts

In accordance with clause MCC 52.232-72, "Limitation of Funds – Incrementally Funded Contracts," funds in the amount of

[TBD] have been allotted to this contract. It is contemplated that funds now obligated under this contract will cover the work to be performed until [TBD]. If the contractor considers the funds obligated under this contract to be insufficient to cover the work to be performed until the date specified herein, or another date agreed to by the parties, the contractor shall notify the contracting officer in writing and indicate the date on which it expects expended funds to approximate 75 percent of the total amount obligated. The notice shall state the estimated amount of additional funds required to continue performance through the date for which incremental funds are provided.

The Contractor shall not perform work on the contract which exceeds the total amount actually allotted by the Government to the contract. The Government is not obligated to reimburse the Contractor for costs incurred in excess of the total amount allotted by the Government to this contract. The contractor assumes the risk for any increased costs beyond what the Government obligates.

B.5. Labor Rates & Other Direct Costs (ODCs)

Labor rates at the inception of a contract period will carry through as "fixed-price" without adjustment for the life of that contract period.

Hours by labor category and pricing under ODCs are estimates only and may be increased/decreased as necessary for successful completion of the contract requirements and as approved by the COR/PM. However, the contractor's expenditures at any given point shall not exceed the total obligated amount or the total ceiling price for this contract as specified below.

Using the template below, Offerors must include all proposed labor categories, fully-burdened hourly rates, Other Direct Costs (ODCs) and G&A information for each contract period identified in the CLIN tables above.

Note that for the Base Period (CLIN 0001), although pricing is being requested in a time-and-materials (T&M) format (to understand the breakdown), this will be a firm-fixed price CLIN; the contractor will be paid the agreed-upon FFP at the conclusion of the Base Period.

Base Option 1 Option 2 Option 3 TOTAL

Total Labor Hours

Total Labor Costs $ - $ - $ - $ - $ -

Total ODCs $ - $ - $ - $ - $ -

Total Est. Cost $ - $ - $ - $ - $ -

Base Period (FFP) – Period of Performance TBD (est. 6 months)

Item # Description(2) U/I Qty Rate Total

LABOR

0001 hr.

hr.

hr.

hr.

Total Labor Hours - -

Labor Sub-Total

ODCs(1)

Base Period (FFP) – Period of Performance TBD (est. 6 months)

Item # Description(2) U/I Qty Rate Total

ODC Sub-Total

Total Firm-Fixed Price (Base Period)

Option Period I (T&M) – Period of Performance TBD (est. 17 months)

Item # Description(2) U/I Qty Rate Total

LABOR

1001 hr.

hr.

hr.

hr.

Total Labor Hours - -

Labor Sub-Total

ODCs(1)

1001 Subcontract(s) for data collection

Lot 1 $150,000.00 $150,000.00

ODC Sub-Total

Total Estimated Value (Option Period I)

Option Period II (T&M) – Period of Performance TBD (est. 12 months)

Item # Description(2) U/I Qty Rate Total

LABOR

2001 hr.

hr.

hr.

hr.

Total Labor Hours - -

Labor Sub-Total

ODCs(1)

2001 Subcontract(s) for data collection

Lot 1 $50,000.00 $50,000.00

ODC Sub-Total

Total Estimated Value (Option Period II)

Option Period III (T&M) – Period of Performance TBD (est. 24 months)

Item # Description(2) U/I Qty Rate Total

LABOR

3001 hr.

hr.

hr.

hr.

Total Labor Hours - -

Labor Sub-Total

ODCs(1)

3001 Subcontract(s) for data collection

Lot 1 $150,000.00 $150,000.00

ODC Sub-Total

Total Estimated Value (Option Period III)

TOTAL COST SUMMARY

Description(2) U/I Qty Rate Total

LABOR

hr.

hr.

hr.

hr.

Total Labor Hours - -

Labor Sub-Total

ODCs(1)

Subcontract(s) for data collection

Lot 2 $150,000.00 $300,000.00

Subcontract(s) for data collection

Lot 1 $50,000.00 $50,000.00

ODC Sub-Total

Total Estimated Cost

NOTE (1): MCC anticipates approximately $350,000.00 worth of data collection subcontracting costs for which the requirements are unknown at this time. To be fair to all vendors providing a quotation, a plug number is utilized for data collection subcontracting costs in the respective periods in which data collection is expected to occur. However, if you believe this plug number is inadequate, please advise during the Question and Answer period of the solicitation. " Subcontract(s) for data collection," as used in the tables above, refers only to the cost of subcontracting a data collection firm and any associated indirect costs (e.g., subcontract handling fees). It does not include any prime contractor labor associated with overseeing/managing the data collection effort; this time should be incorporated into the level of effort proposed under the respective Labor CLIN. Subcontracting costs for data collection are subject to the requirements of FAR 52.244-2, Subcontracts.

NOTE (2): A job description shall be included for each labor category proposed. The job description must identify essential and non-essential tasks that are assigned to a specific position.

It must also identify reporting relationships and shall also describe required qualifications, minimum requirements, working conditions and desirable qualifications.

(End of SECTION B)

Section C - Description/Specifications/Work Statement

Statement of Work

SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK

The Millennium Challenge Corporation (MCC) is a U.S. Government corporation whose mission is to provide assistance that will support economic growth and poverty reduction in carefully selected countries that demonstrate a commitment to just and democratic governance, economic freedom, and investments in their citizenry. For more information on the MCC, please visit www.mcc.gov. The MCC's Monitoring and Evaluation (M&E) Policy can accessed here:

https://www.mcc.gov/resources/doc/policy-for-monitoring-and-evaluation.

C.1. Scope

Provide Independent Evaluation Services in support of the Policy Reform and Institutional Strengthening (PRIS) Project. This includes all three activities under this Project: (1) the Policy, Regulation, and Institutional Support Activity; (2) the Utility Strengthening Activity; and (3) the Public Information and Education Activity.

C.2. Introduction and Background

The Benin II Compact between the MCC and the Government of Benin (GoB) was signed on September 9, 2015. The Compact Objective is to expand business production and productivity, generate greater economic opportunities for households and improve the capacity to provide public and social services by improving the quantity and quality of the supply of electricity in Benin. The Benin II Compact (the "Program") entered into force (EIF) on June 22, 2017. A full copy of the Program can be accessed on MCC's website here: https://www.mcc.gov/where-we-work/program/benin-power-compact. The Program's Monitoring and Evaluation Plan is available here:

https://www.mcc.gov/where-we-work/m-and-e/benin-power-compact.

The Program targets poor electricity infrastructure, a binding constraint to economic growth in Benin, where the insufficient quantity and quality of electricity results in reduced productivity, output, and investment for businesses, less effective delivery of public and social services and diminished well-being and economic opportunity for households. The root causes of these problems are lack of resources, and public policies and institutions that have been unable to deliver sufficient power to meet growing national demand.

The Program will address these core problems through policy reforms and institutional strengthening, large-scale investments in electricity generation and distribution infrastructure, energy efficiency, and off-grid electrification to expand access in a country where only one-third of the population has access to electricity. As a result of these interventions, the Program is expected to expand business production and productivity, increase economic opportunities for households, and improve the capacity to provide public and social services.

The Compact includes four Projects, each with a related project objective:

• The Electricity Generation Project ($135.87 Million), with the objectives of: (A) increasing the hours of operation for businesses and public and social services; (B) reducing reliance on costlier sources of energy; (C) reducing losses of products and perishable goods; and (D) improving productivity for users of electricity. It is comprised of the Photovoltaic Generation and Hydroelectric Generation activities. (The Compact initially included a Thermal Generation Activity ($12.475 Million). This was removed from MCC's investment as a result of the Government of Benin moving forward with the rehabilitation of the three thermal generating units using national funds.)

• The Electricity Distribution Project ($109.71 Million), with the objectives of: (A) increasing the hours of operation for businesses and public and social services, (B) reducing reliance on costlier sources of energy, (C) reducing losses of products and perishable goods, and (D) improving productivity for users of electricity. It is comprised of the Regional Grid Strengthening, Cotonou Grid Strengthening, and National Electricity Dispatch activities.

• The Off-Grid Electricity Access Project (OGEAP) ($45.95 Million), with the objective of increasing access to electricity and thereby (A) increasing the hours of operation for businesses and public and social services, (B) reducing reliance on costlier sources of energy, (C) reducing losses of products and perishable goods, and (D) improving productivity for users of electricity. It is comprised of the Enabling Environment for Off-Grid Electricity and Off-Grid Clean Energy Facility (OCEF) activities.

• The Policy Reform and Institutional Strengthening (PRIS) Project ($41.23 Million), with the objective of strengthening the capacity of the Société Béninoise d'Énergie Électrique (SBEE), the Autorité de Régulation de l' Électricité (ARE), and other Government of Benin institutions to improve the governance, management, and operations of the electricity sector. It is comprised of the following three activities: (1) Policy, Regulation, and Institutional Support; (2) Utility Strengthening; and, (3) Public Information and Education. As they are the subject of this evaluation, these three activities are described in further detail below.

The program logic contained in Figure 1,Policy Reform and Institutional Strengthening: Policy, Regulation, and Institutional Support Activity (see Attachment J-11), outlines the anticipated logic and theory of change for the Program.

The Policy, Regulation, and Institutional Support Activity ($21.91 Million) aims to improve the overall governance of the electricity sector in Benin by improving regulation; establishing a tariff policy, plan, and periodic tariff adjustments; increasing energy efficiency; and strengthening the policy and institutional framework for Independent Power Producers (IPPs). MCC Funding will support:

(A) Assistance to ARE through operational support and capacity building; establishment of regulatory tools, processes, and procedures; preparation and implementation of tariff studies and development of a tariff policy and Tariff Plan to achieve cost-reflective tariffs, encouragement of private sector investment in power generation, and ensuring access for the poor;

(B) Expansion and strengthening of energy efficiency standards and labeling programs in Benin, including the formulation of technical standards, development of a program for product labeling, enhancement of product quality enforcement and testing, introduction of support for public sector procurement of energy efficient goods and industrial energy efficiency;

(C) Improvement of the environment for IPPs, including by establishing a legal and regulatory framework, standard forms of contract, credit enhancement mechanisms and a competitive solicitation process by means of technical assistance, transaction advisory services and other support; and

(D) Support of the following policy reviews: legal review of the electricity codes (the Benin-Togo and the Benin Electricity Codes) to target necessary revisions; technical analysis of structuring options for Government and SBEE -owned generation assets including creation of stand-alone enterprises; and situation assessment and preliminary study as input to national gas policy and strategy for acquiring a reliable supply of gas for power generation.

The program logic contained in Figure 2, Policy Reform and Institutional Strengthening: Utility Strengthening Activity (see Attachment J-11), outlines the anticipated logic and theory of change for the Policy, Regulation, and Institutional Support Activity:

The Utility Strengthening Activity ($17.32 Million) aims to ensure the operational independence of SBEE and improve its core business functions so as to become more financially viable and better able to serve its customers. Together with the sector-wide improvements from the Policy, Regulation and Institutional Support Activity, the Utility Strengthening Activity is expected to help improve key aspects of SBEE's operations and to result in its ability to better provide power to consumers of all income classes. MCC Funding will support:

(A) Implementation of governance and management reforms of SBEE, including through technical and other advisory services and capacity building for corporate and strategic management;

(B) Support for a Management Services Contract to improve the management and operations of SBEE in order to achieve key performance indicators (TBD) within a four-year period and to establish standard operating procedures, deliver training, mentoring and coaching, and other measures to sustainably improve the capacity and effectiveness of SBEE staff. (The Compact's original design only included a Contrat Plan (performance contract between the government and the utility), which was validated in June 2017. The Government of Benin requested that MCC go further with in its support of electricity sector reform in Benin and establish a management services contract. As a result, MCC requested and obtained approval from its Investment Management Committee to redesign the Compact accordingly.)

(C) Strengthening of SBEE's maintenance capabilities including development of a maintenance management system and acquisition of related software and standard operating procedures; and

(D) Capacity building for SBEE personnel through training and instituting policy and procedural changes to create a safe working environment (including with respect to protections from sexual harassment) with equal opportunities for men and women's professional advancement.

The Management Services Contract is expected to be awarded in May 2019.

The program logic contained in Figure 3, Policy Reform and Institutional Strengthening: Public Information and Education Activity (see Attachment J-11), outlines the anticipated logic and theory of change for the Utility Strengthening Activity:

The Public Information and Education Activity ($2 Million) aims to inform the public about energy sector policy reforms and to change energy use behavior through information and education about energy efficiency, renewable electricity sources and related topics. To this end, the Public Information and Education Activity will create a program to educate consumers about energy efficiency and renewable energy for household use. The Public Information and Education Activity will take into account language differences, education levels, gender and rural and urban access to information and technology.

The program logic contained in Figure 4, Policy Reform and Institutional Strengthening Program Logic: Public Information and Education Activity (see Attachment J-11), outlines the anticipated logic and theory of change for the Public Information and Education Activity:

The overall risks and assumptions of the Project include, but are not limited to:

• The GoB will be able (and willing) to effectively enforce energy efficiency standards, despite porous land borders where smuggling is rife.

• An improved IPP framework will be sufficient for businesses to overcome other impediments to investing in Benin, particularly the constraints to the overall business environment that were identified in the constraints analysis.

• Businesses will have sufficient assurance and trust that SBEE will pay for the electricity they produce.

• The sector's regulator will not only be implemented, but will be implemented in a way that maintains its independence from political and outside influence.

• Independence in energy sector governance will be not only be necessary, but also sufficient to improve the overall effectiveness of sector governance.

• All technical assistance and training will be fully implemented within SBEE, such that systems will be maintained and procedures followed even after the end of the compact, despite potential resistance from some actors within

SBEE.

C.3. Evaluation Questions and Guidance on Methodology

A. Evaluation Questions

The anticipated research questions are listed below. The Contractor will be expected to work with MCC, the Millennium Challenge Account (MCA) Benin II, and local stakeholders to assess and finalize this list and to propose appropriate methods for responding to these questions as a part of the elaboration of the Evaluation Design Report (EDR).

To the extent possible and relevant, answers to evaluation questions shall include:

• An assessment of why or why not changes were observed (e.g. political economy factors, incentives, implementation factors).

• A discussion of major unanticipated outcomes and externalities (positive or negative).

• A discussion of whether the observed changes are likely to be sustainable.

• An analysis of how (and why) effects may vary by gender, geographic location, socio-economic status, or other characteristics of beneficiaries and participants.

• A discussion of major external factors that enabled or hindered the attainment of project outcomes.

The Contractor's methodology shall describe clearly how the evaluation will address these questions.

Project-wide

1. What are:

a. The quality of project design?

b. The soundness of the program logic?

c. The fidelity and degree of program implementation? In the event of deviations from the original design

(e.g., in terms of objectives, activities, or beneficiaries), why did they occur and what were the implications for overall outcomes and intended results?

d. The lessons learned?

2. Were the sub-activities timed and sequenced in such a manner to facilitate the achievement of expected results?

3. What was the project's economic impact?

4. Was the project cost-effective?

Policy, Regulatory, and Institutional Support

Energy efficiency

1. To what extent has the Government of Benin adopted and implemented policies and actions to improve energy efficiency?

2. To what extent were new or strengthened standards and labeling for energy efficiency implemented during the Compact? To what extent were those standards enforced fairly and efficiently?

3. To what extent have retailers begun selling energy-efficient labeled merchandise? Has the proportion of energy efficient vs. non-energy efficient products on the market changed in terms of availability and sales?

4. To what extent do consumers recognize and make purchasing decisions on the basis of the new labeling system?

5. To what extent have consumer savings resulted from these policies and actions?

Independent Power Producers (IPPs)

6. To what extent were new policies and frameworks for IPPs implemented?

7. Have any IPP transactions reached financial close (Definition: The period of time after an agreement has been entered when all conditions have been fulfilled (or waived) and all documents have been properly filed and executed. After the financial close, drawdowns are allowed.)?

8. How much private investment is there in IPP power generation in Benin?

9. What percentage of Benin's electricity consumption is produced by IPPs and from clean energy sources?

10. Are the Power Purchase Agreements (PPAs) and associated agreements in place being respected? Is the utility paying the IPPs on time? Have any government guarantees been drawn on as a result of non-payment? Are there any arbitrations or legal proceedings between the parties to an IPP transaction?

11. Do IPPs perceive the regulatory framework as credible and transparent?

Tariff policy

12. To what extent has the new tariff policy been implemented? To what extent do electricity tariffs in Benin reflect the cost of service?

13. Has the sector regulator assumed its mandated role in setting and adjusting tariffs?

14. What is the level of public awareness and acceptance of the new tariffs among the different categories of households, businesses, and public services?

15. Are the structures and procedures in place to allow recurring adjustments to the tariff, such that it will be able to remain cost-reflective into the future?

16. How has the new tariff structure affected SBEE's balance sheet, income statement, and cash flow statement?

17. To what extent has increased internally-generated cash flow (if any) allowed SBEE to finance increased maintenance or network expansion? What changes (if any) has SBEE experienced or implemented as a result?

18. Did the automatic tariff adjustment tool generate tariff adjustments reflecting changes in inflation, fuel costs, and exchange rates? Within the Compact period, to what extent were tariff changes driven by reforms to the tariff structure and methodology versus other factors (e.g., inflation, cost of fuel exchange rates)?

Master Plan and updating of the Energy Code

19. To what extent have the new policies been implemented?

20. How has overall planning and coordination among actors within the sector changed?

21. What achievements (if any) have been made as a result of changes in sector planning?

Operational capacity and independence of the regulator (ARE)

22. To what extent is ARE operational? Does ARE have the financial and human resources necessary to successfully carry out its mandate?

23. Does ARE have the data necessary to measure sector performance and conduct technical analyses? If so, do those data influence ARE decisions?

24. To what extent hasARE been able to make major decisions independently from the Government?

25. As a result of any changes (if any) in the level of independence of decision making, what changes in sector functioning have occurred? What have the changes enabled SBEE (or other actors in the sector) to do as a result?

What observable effects (if any) have ARE decisions had on the financial health of SBEE?

26. How do private sector actors perceive ARE's ability to carry its mandate? How has that perception influenced private sector participation in the sector?

Utility Strengthening

1. To what extent has SBEE been able to make major decisions independently from the Government?

2. To what extent have GoB and SBEE abided by the terms of the approved Contrat Plan year by year since its adoption?

3. Has the management services contract been put into place at SBEE? Has the management services contractor been able to meet its commitments under the management contract? What performance improvements have been achieved during the term of the management services contractor? How has the management services contractor performed against the KPIs in the management contract? Has the management services contractor provided the training and capacity building of the local management of SBEE?

4. What are the perceptions (by GoB, SBEE employees, customers, and other stakeholders) of the performance of the management services contractor?

5. How do independent power producers (IPPs) perceive SBEE's ability to meet its obligations under power purchase agreements (PPAs)? How has that perception influenced private sector participation in the sector? How has that perception influenced the need for government backstops to SBEE obligations under the PPAs?

6. Did SBEE's cost recovery and financial health improve?

7. To what extent did SBEE's billing and payment processes improve from the perspective of its personnel and of its customers?

8. Did SBEE improve its bill collection and reduce its overall commercial losses? To what extent did the reduction in commercial losses (if any) increase the efficient allocation of electricity between users?

9. To what extent has the utility changed its staffing and staff management? To what extent have these changes increased the utility's labor productivity?

10. How accurate and updated is the new asset management system? How successful is SBEE in implementing and sustaining an evidence-based, lifecycle-centric approach to asset management? Does SBEE continue to use and update the system even after the end of the Compact?

11. Did the technical assistance to SBEE lead to improved maintenance practices?

12. Does SBEE have the technical and financial capacity to continue maintaining infrastructure (both MCC and non-MCC funded)?

13. What is the total cost savings for SBEE as a result of improved maintenance? (How) is this cost savings put to use to improve the financial or operational health of the utility?

14. Did improved maintenance lead to technical loss reduction? If so, to what extent?

15. Did improved maintenance increase electricity quality (i.e. increase voltage stability and reduce unplanned and network maintenance-related outages)? If so, to what extent?

16. In what other ways have SBEE management practices changed? Are these changes associated with more efficient operations?

17. What was the total amount of cost savings achieved by operational area discussed above and by year? Are these savings sustainable?

18. What was the total improvement in electricity quality achieved by type and by year? Are these improvements sustainable?

Public Information and Education

1. To what extent did the public information and education campaigns reach their intended audience? Did the intended audience understand the campaigns' content as intended? Did they retain the information?

2. Were beneficiaries of the public information and educationcampaigns more likely to purchase energy efficiency technologies and adopt energy efficiency behaviors than those who were not reached? Did their beliefs concerning energy efficient technologies and behaviors change?

3. Were beneficiaries of the public information and education campaigns more likely to accept the tariff changes than those who were not reached?

B. Guidance on Methodology

MCC procures two different types of evaluations: Impact Evaluations and Performance Evaluations. (MCC defines an impact evaluation as a study that measures the changes in income and/or other aspects of well-being that are attributable to a defined intervention. Impact evaluations require a credible and rigorously defined counterfactual, which estimates what would have happened to the beneficiaries absent the project. MCC defines a performance evaluation as a study that seeks to answer descriptive questions, such as: what were the objectives of a particular project or program; what the project or program has achieved; how it has been implemented; how it is perceived and valued; whether expected results are occurring and are sustainable; and other questions that are pertinent to program design, management, and operational decision making . MCC's performance evaluations also address questions of program impact and cost-effectiveness.) MCC does not expect an Impact Evaluation for the PRIS project as a whole because of the absence of a valid counterfactual. Nevertheless, MCC is receptive to Impact Evaluation methodologies for specific activities or sub-activities, such as the Public Information and Education sub-activity. The Contractor shall carefully consider the tradeoff between the potential for learning and the anticipated evaluation cost when proposing such an approach.

MCC recognizes the challenges to evaluating policy and institutional reform projects. The guidance provided below to address those challenges should not be construed as comprehensive nor conclusive. The Contractor should think creatively about how diverse research methods could be used to assess project outcomes in a cost-effective and rigorous manner.

Integration: This evaluation covers the Project's seven sub-activities. The Contractor shall develop distinct methodologies for each sub-activity as warranted within an integrated approach covering the evaluation. Data collection and analysis should be conducted across sub-activities to the extent possible. The Contractor's methodology shall ensure the interaction of the PRIS sub-activities are addressed.

Prioritization: The research questions listed above suggest a large scope of the evaluation. The Contractor's methodology shall prioritize questions based on their relevance to the project logic, the value of learning that would be generated, the anticipated data collection costs, the likelihood that results would be observable within the evaluation timeframe, and other factors the Contractor deems pertinent. The Contractor shall also prioritize based on available data including administrative data, project monitoring data, MCA-led surveys, key performance indicator reporting from the management contractor, and data collected from the other independent evaluations.

Implementation analysis: MCC expects the evaluation to include an analysis of implementation processes. This shall go beyond a standard process evaluation assessing whether the project was implemented according to plan. Rather, the Contractor shall provide a more in-depth analysis of the implementation factors that enabled or hindered the attainment of anticipated outcomes. These factors may include sequencing of sub-activities, flexibility in implementation, communications, relationships, incentives, power dynamics, quality of consultant deliverables, and political economy factors. This aspect of the evaluation is expected to contribute to learning on implementing policy and institutional reform projects across sectors and countries.

Attribution: The Contractor's methodology should clarify the extent to which observable outcomes are attributable to the PRIS Project. Reform activities often enable or facilitate the achievement of desired outcomes without directly causing them.

The Contractor's methodology should include a discussion of how other interventions within and outside of the Compact influenced project results.

Cost-effectiveness: The Contractor should develop a methodology to assess the project's economic impact and/or cost-effectiveness. The Contractor's approach may include the use of cost-benefit analysis (CBA) to independently estimate parts of or the entire PRIS project's economic rate of return(ERR) to provide insights into the magnitude and distribution of economic benefits. However, the Contractor may propose other forms of cost-effectiveness analysis, as they see fit. MCC did not estimate a distinct ERR for the PRIS Project, although the full costs of the PRIS Project, and portions of its benefits, are included in MCC's Original CBA model for the overall on-grid energy investments. Therefore, MCC's policy of conducting ex-post independent reviews of its cost-benefit analysesis not possible and does not apply to this evaluation.

Scale and scope of baseline and interim evaluation: The Contractor's methodology should include a baseline report being submitted in Year 3 of the Program/Compact and an interim evaluation report being submitted in Year 5 of the Program/Compact. MCC expects less data collection and overall level of effort for the baseline and interim evaluation than for the final evaluation report. These are expected to be largely based on existing data sources and qualitative data collection.

C. Data Considerations and Availability

MCC and MCA Benin-II will provide documentation and data for the evaluation to the Contractor. This may include, but is not be limited to, the following:

• Progress reports from MCA Benin-II, the Management Services Contractor, and/or other relevant implementing entities

• Indicator Tracking Tables (ITTs) and narrative reports

• Data Quality Review reports

• Scope of work, methodological documents, and/or reports from MCA-managed studies including the baseline study on selected indicators, the SBEE staff satisfaction survey, and the survey on tariff acceptance and awareness

• Deliverables from the transaction advisor, such as the SBEE diagnostic report, the investment and turnaround management action plan for SBEE, and the transaction structure report

• SBEE's Annual Work Plan and Budget

• Contrat Plan between SBEE and the Government of Benin

• Audit report(s) of the Contrat Plan and the Management Services Contract

• Energy efficiency audit methodological documents and report(s)

• Sector master plan

• Tariff Study Report

• Documentation related to the number of Power Purchase Agreements closed and megawatts of generation capacity added to the system as a result of the Compact

• Communication plans under the Public Information and Education Activity

• Final report on the feasibility of creating a power generation company of Benin (GENCO) Data and documentation from the other Benin-II independent evaluations, many of which will be publically available on the MCC Evaluation Catalog (https://data.mcc.gov/evaluations/index.php/catalog?&page=2).

Many of the documents listed above are not yet available.

C.4. Roles and Responsibilities

A. Independence:

MCC's Policy on Independent Evaluations. Based on the 1991 DAC Principles for Evaluation of Development Assistance (http://www.oecd.org/dac/evaluation/dcdndep/45438179.pdf), MCC's Evaluation Policy requires independent evaluations of each MCC project. MCC's implementation of this independence policy adheres to these principles, especially the following: " Impartiality and independence will best be achieved by separating the evaluation function from the line management responsible for planning and managing development assistance." (P.6.)

Organizational Conflicts of Interest. MCC-M&E is responsible for adhering to our policy-driven requirements for independence. These requirements are distinct from Organizational Conflicts of Interest (OCI) proscribed in the Federal Acquisition Regulations. Interpretations and enforcement of the OCI statutes are the responsibility of our Office of General Counsel.

Independence of Evaluation Team. Evaluation team members must be impartial and therefore should not have contributed to the design or implementation of the project being evaluated. In cases where individuals or firms that have worked on the project are proposed to be part of the evaluation team, a justification and independence risk mitigation plan must be provided to MCC for consideration.

The Independent Evaluator's Role. To remain consistent with the principles of independent evaluations during project planning and implementation, the independent contractor's feedback is restricted to reporting evidence and findings.

Recommendations shall not be made before completing a final evaluation as they can influence project planning or implementation, thereby jeopardizing the independence an evaluation. Prescriptions and recommendations may be included in final evaluations.

Real-time feedback to project implementation. MCC supports monitoring and evaluation activities intended to provide feedback on how to improve program planning and implementation. But this support shall be separate from the independent contractor. Any contribution of the Independent Contractor to this feedback shall be limited to reporting evidence and findings.

Any exceptions to this rule must be approved in advance by the Evaluation Management Committee (EMC).

B. MCC & MCA Responsibilities: The MCC, MCA, along with the Independent Evaluator, play critical roles in designing, implementing, and disseminating the independent evaluations. The MCC and MCA roles are as follows (For further detail see

Section J.1: Evaluation Management and Review Process):

1. Millennium Challenge Corporation (MCC): MCC is responsible for oversight of the Independent Contractor and quality control of evaluation activities, including the following specific responsibilities:

a. Assess when a program is ready for evaluation planning through evaluability assessment (An assessment of the extent to which an intervention can be evaluated in a reliable and credible fashion.);

b. Determine what program components (Project(s), Activities, Sub-Activities) will be covered by the evaluation;

c. Set the evaluation questions to achieve accountability and learning objectives;

d. Build buy-in and ownership of the evaluation;

e. Contract and Supervise the Independent Evaluator;

f. Conduct quality reviews of all evaluation products (reports, questionnaires, etc.);

g. Facilitate public dissemination efforts to inform decision-makers on learning generated by the evaluation;

h. Oversee preparation of public-use data that ensures appropriate balance of enabling verification of analysis and broad use of the data with adherence to promises of confidentiality to survey respondents;

and

i. Determine the evaluation budgets.

2. Millennium Challenge Account (MCA) and country partners: MCA and country partners are responsible for building local ownership and commitment to the evaluation, oversight of the data collection firm, and quality control of evaluation activities, including the following specific responsibilities:

a. Assess when a program is ready for evaluation planning through evaluability assessment;

b. Set the evaluation questions to achieve accountability and learning objectives;

c. Build buy-in and ownership of the evaluation;

d. Conduct quality reviews of all evaluation products (reports, questionnaires, etc.); and

e. Facilitate public dissemination efforts.

C.5. Tasks

Independent Evaluator (hereafter referred to as the Contractor): The Contractor shall be responsible for the overall design, implementation, and dissemination of the evaluation, including the following specific responsibilities:

a. Develop the most rigorous evaluation design feasible given rules of program implementation

b. Support MCC and MCA to build buy-in and ownership for the evaluation.

c. Develop evaluation materials that are held to international standards

d. Ensure Institutional Review Board clearance of survey protocols

e. Manage the data collection firms

f. Ensure data quality during collection and entry through supervisions and management

g. Lead data cleaning, analysis, and interpretation of results

h. Produce high quality, credible, transparent evaluation reports

i. Lead preparation of public-use data that ensures appropriate balance of enabling verification of analysis and broad use of the data with adherence to promises of confidentiality to survey respondents

j. Lead public dissemination efforts

The following section details the required sub-tasks for each evaluation task. MCC anticipates there being additional sub-tasks that may be required based on the evaluation design

C.5.1 Task 1: Assess evaluation plan

• The Contractor shall be responsible for conducting an Evaluability Assessment on all existing documentation related to the program and proposed evaluation plan provided by MCC and MCA using the MCC Evaluability Assessment Tool (See Section J) as a guide.

• MCC and the Contractor should work together to identify all MCC and MCA self-evaluation activities and determine how the independent evaluation will (i) validate findings of self-evaluation(s), (ii) use self-evaluation(s) to inform the independent evaluation, particularly around understanding what was designed, what was implemented, and understanding any necessary course corrections, (iii) complement the self-evaluation(s) by expanding on self-evaluation activities.

• The Contractor shall produce an Evaluability Assessment Report that provides an independent assessment of the program being evaluated. This assessment should be guided by the Evaluability Assessment Tool and should assess the quality, completeness, and/or appropriateness of the Problem Diagnostic, Program Logic, Cost-Benefit Analysis, Risks & Assumptions, Monitoring Indicators, and Proposed Evaluation Questions. The report should aim to convey the Contractor's overarching assessment of the coherence of the program design, the underlying economic analysis, and the M&E approach.

• Develop a Work Plan with expected deadlines for deliverables. Any revisions to the Work Plan shall be submitted in the Contractor's Progress Reports.

C.5.2 Task 2: Develop evaluation design report

• The Contractor shall conduct missions to the field, as needed, to discuss program design and implementation issues to inform the evaluation design, as well as engage local stakeholders (MCA and Implementing Entities, as well as local government) on the proposed evaluation. This includes hosting local stakeholder workshops to develop/refine research questions, assess appropriate and feasible quantitative and qualitative evaluation methodologies, develop evaluation timeline, and build local stakeholder commitment to the evaluation design.

• The Contractor shall develop the Evaluation Design Report, including all technical analysis required for assessing and confirming (as necessary) internal validity, power calculations, and required sample sizes. The Evaluation Design Report outline shall be developed in consultation with the MCC M&E lead. Please reference the Evaluation Design Report Outline (See Section J) as a guide.

• Per MCC's Gender Policy and Gender Integration Guidelines, the Contractor shall ensure that evaluation design, data collection requirements, and analysis address gender and other social differences as appropriate. This involves including research questions that address social and gender dimensions of the project logic, assumptions, and project design; disaggregating impacts by gender, age, income, and other social difference (e.g. ethnicity) as appropriate; including special modules or studies as needed.

• The Contractor should propose a methodology for assessing the project's cost-effectiveness. This proposal may include a CBA methodology to estimate the project's Evaluation-Based ERR, or it may adopt another form of cost-effectiveness analysis.

• The Contractor shall document feedback from local stakeholders and the response to that feedback in an annex to the final report. In order to obtain feedback, the Contractor may lead local workshops and/or virtual reviews of the Design Report by MCA and Implementing Entities.

a. In accordance with Section H, the Contractor is responsible for allowing any key local stakeholders the opportunity to submit a public Statement of Difference or Support, as agreed with MCC and the MCA/designated representative.

• The Contractor shall submit the Evaluation Design Report to MCC and may be asked to present to the Evaluation Management Committee (See Section J). All feedback and response to feedback will be documented by the Contractor in an annex to the final report.

a. In accordance with Section H, MCC may: (i) select to obtain external peer review; and (ii) make the Report and peer reviews public on the MCC website.

• The Contractor shall produce a Nesstar Metadata file (See Section J) summarizing the final Evaluation Design Report for dissemination on the MCC Evaluation Catalog. Prior to publication, the Contractor should review the current version of MCC's Evaluation Microdata Guidelines (See Section J) for guidance on preparation of public documentation that links to microdata to reduce re-identification risk.

• The Contractor shall continue to maintain the Evaluation Design Report as…