RFQ_95332418Q0109_Benin_II_OGEAP_Evaluation.pdf

PAGE 1 OF

1. REQUISITION NO.

2. CONTRACT NO. 3. AWARD/EFFECTIVE DATE 4. ORDER NO. 5. SOLICITATION NUMBER 6. SOLICITATION ISSUE DATE

a. NAME b. TELEPHONE NO. (No Collect Calls) 8. OFFER DUE DATE/LOCAL

9. ISSUED BY CODE 10. THIS ACQUISITION IS UNRESTRICTED OR SET ASIDE: % FOR:

SMALL BUSINESS

HUBZONE SMALL

BUSINESS

SERVICE-DISABLED

VETERAN-OWNED

SMALL BUSINESS

WOMEN-OWNED SMALL BUSINESS

(WOSB) ELIGIBLE UNDER THE WOMEN-OWNED

SMALL BUSINESS PROGRAM

EDWOSB

8(A)

NAICS:

SIZE STANDARD:

11. DELIVERY FOR FOB DESTINA-

TION UNLESS BLOCK IS

MARKED

SEE SCHEDULE

12. DISCOUNT TERMS

13a. THIS CONTRACT IS A

RATED ORDER UNDER

DPAS (15 CFR 700)

13b. RATING

14. METHOD OF SOLICITATION

RFQ IFB RFP

15. DELIVER TO CODE 16. ADMINISTERED BY CODE

17a. CONTRACTOR/OFFEROR CODE FACILITY CODE 18a. PAYMENT WILL BE MADE BY CODE

TELEPHONE NO. DUNS: DUNS+4:

PHONE: FAX:

17b. CHECK IF REMITTANCE IS DIFFERENT AND PUT SUCH ADDRESS IN OFFER

18b. SUBMIT INVOICES TO ADDRESS SHOWN IN BLOCK 18a UNLESS BLOCK BELOW IS CHECKED

SEE ADDENDUM

19. 20. 21. 22. 23. 24.

ITEM NO. SCHEDULE OF SUPPLIES/SERVICES QUANTITY UNIT UNIT PRICE AMOUNT

(Use Reverse and/or Attach Additional Sheets as Necessary)

25. ACCOUNTING AND APPROPRIATION DATA 26. TOTAL AWARD AMOUNT (For Govt. Use Only)

27a. SOLICITATION INCORPORATES BY REFERENCE FAR 52.212-1, 52.212-4. FAR 52.212-3 AND 52.212-5 ARE ATTACHED. ADDENDA ARE ARE NOT ATTACHED.

27b. CONTRACT/PURCHASE ORDER INCORPORATES BY REFERENCE FAR 52.212-4. FAR 52.212-5 IS ATTACHED. ADDENDA ARE ARE NOT ATTACHED

28. CONTRACTOR IS REQUIRED TO SIGN THIS DOCUMENT AND RETURN _______________ 29. AWARD OF CONTRACT: REF. ___________________________________ OFFER

COPIES TO ISSUING OFFICE. CONTRACTOR AGREES TO FURNISH AND DATED ________________________________. YOUR OFFER ON SOLICITATION

DELIVER ALL ITEMS SET FORTH OR OTHERWISE IDENTIFIED ABOVE AND ON ANY (BLOCK 5), INCLUDING ANY ADDITIONS OR CHANGES WHICH ARE

ADDITIONAL SHEETS SUBJECT TO THE TERMS AND CONDITIONS SPECIFIED SET FORTH HEREIN IS ACCEPTED AS TO ITEMS:

30a. SIGNATURE OF OFFEROR/CONTRACTOR 31a. UNITED STATES OF AMERICA (SIGNATURE OF CONTRACTING OFFICER)

30b. NAME AND TITLE OF SIGNER (TYPE OR PRINT) 30c. DATE SIGNED 31b. NAME OF CONTRACTING OFFICER (TYPE OR PRINT) 31c. DATE SIGNED

AUTHORIZED FOR LOCAL REPRODUCTION (REV. 2/2012)

PREVIOUS EDITION IS NOT USABLE Prescribed by GSA - FAR (48 CFR) 53.212

7. FOR SOLICITATION

INFORMATION CALL:

STANDARD FORM 1449

OFFEROR TO COMPLETE BLOCKS 12, 17, 23, 24, & 30

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL ITEMS 75

95332418Q0109 06/27/2018

David Fuller (Contractor) fullerdf@mcc.gov TIME Noon ET

07/25/2018

Millennium Challenge Corporation

Contracts and Grants Management Division

1099 Fourteenth Street NW

Suite 700, Attn: David Fuller (Contractor) fullerdf@mcc.gov

Washington DC 20005

X

X

N/A

X

Millennium Challenge Corporation

CGM

1099 Fourteenth Street NW, Suite 700

Washington DC 20005

Millennium Challenge Corporation

Contracts and Grants Management Division

1099 Fourteenth Street NW

Suite 700

Washington DC 20005

Interior Business Center

M/S D-2773

7301 West Mansfield Avenue

Lakewood CO 80235-2230

This is a combined synopsis/solicitation for commercial items prepared in accordance with the format in FAR Subpart 12.6 and conducted per the procedures of FAR Subpart 13.5, as supplemented with additional information included in this notice.

MCC seeks a Contractor to provide Independent Evaluation

Services in support of Benin II Off-Grid Energy Access Project.

See enclosed full and open competitive solicitation and attachments for pertinent associated attachments for all details.

See CONTINUATION Page

X X

Jacqueline H. Naranjo

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 2

Table of Contents

SECTION B – SUPPLIES OR SERVICES AND PRICE/COSTS

B.1. Price/Cost Schedule

B.2. FAR 52.216-1 -- Type of Contract (APR 1984)

B.3. Accounting and Appropriation Data

B.4. Funding Type

B.5. Limitation of Funds – Incrementally Funded Contracts

B.6. Labor Rates & Other Direct Costs (ODCs)

SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK

C.1. Introduction and Background

C.2. Scope

C.3. Evaluation Questions and Guidance on Methodology

C.4. Roles and Responsibilities

C.5. Tasks

SECTION D – PACKAGING AND MARKING

D.1. Disclaimer for Deliverables

SECTION E – INSPECTION AND ACCEPTANCE

E.1. Inspection of Reports

E.2. Quality Control Plan

SECTION F – DELIVERIES OR PERFORMANCE

F.1. Period of Performance

F.2. Place of Performance

F.3. Deliverables

F.4. Reporting Deliverables

F.5. Deliverable Format

F.6. Acceptance Criteria

SECTION G – CONTRACT ADMINISTRATION DATA

G.1. MCC 52.201-70 Contracting Officer’s Representative (COR) and/or Project Monitor (PM) (JULY 2012) .. 30

G.2. MCC 52.232-70 Invoice Instructions (MAY 2017)

SECTION H – SPECIAL CONTRACT REQUIREMENTS

H.1. MCC 52.203-70 Contractor Non-Disclosure Agreement (MAY 2017)

H.2. MCC 52.204-70 Security Requirements for Contractor Personnel (MAY 2013)

H.3. MCC 52.209-70 Organizational Conflicts of Interest: Preclusion from Implementation Contract (JULY 2012)

H.4. MCC 52.232-72 Limitation of Funds – Incrementally Funded Contracts (NOV 2006)

H.5. MCC 52.232-73 Travel Reimbursement (MAY 2017)

H.6. MCC 52.232-74 ODC Reimbursement (JAN 2007)

H.7. MCC 52.242-70 Contractor Performance Assessment Rating System (CPARS) Registration (AUG 2011) .. 41

H.8. MCC 52.243-70 Increase in Services (OCT 2006)

H.9. MCC 52.245-70 Contractor Acquired MCC Government Property Overseas (JULY 2012)

H.10. Provision on Peer Review and Posting of Reports

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 3

H.11. Provision on Insurance Maintenance

H.12. Provision on Stakeholder Input for Evaluation Products

H.13. Provision on Human Subjects Protection

H.14. Personally Identifiable Information and Confidentiality

H.15. Provision on Delivered Work Product Free from 3rd Party Rights

H.16. Key Personnel

H.17. Team & Key Personnel Qualifications

PART II – CONTRACT CLAUSES

SECTION I – CONTRACT CLAUSES

I.1. FAR 52.212-4 Contract Terms and Conditions — Commercial Items/Alternate I (JAN 2017)

I.2. FAR 52.212-5 Contract Terms and Conditions Required to Implement Statutes or Executive Orders –

Commercial Items (JAN 2018)

I.3. FAR 52.214-34 Submission of Offers in the English Language (APR 1991)

I.4. FAR 52.214-35 Submission of Offers in U.S. Currency (APR 1991)

I.5. FAR 52.217-9 Option to Extend the Term of the Contract (MAR 2000)

I.6. FAR 52.252-2 Clauses Incorporated by Reference (FEB 1998)

PART III – LIST OF DOCUMENTS, EXHIBITS AND OTHER ATTACHMENTS

SECTION J – LIST OF ATTACHMENTS

SECTION K – REPRESENTATIONS, CERTIFICATIONS AND OTHER STATEMENTS OF OFFERORS OR

RESPONDENTS

SECTION L – INSTRUCTIONS, CONDITIONS, AND NOTICES TO OFFERORS

L.1. Inquiries/Questions Concerning the Solicitation

L.2. Exclusion of Quotes and Pre-Award Communications with Offerors

L.3. Quotation Due Date

L.4. Quotation Response

L.5. Quotation Volumes

L.6. Quotation Format

L.7. Blocked and Unacceptable Electronic File Types

L.8. Page Limits

L.9. Preparation of Volume I – Technical Quotation

L.10. Preparation of Volume II – Cost Quotation

SECTION M – EVALUATION FACTORS FOR AWARD

M.1. FAR 52.212-2 Evaluation – Commercial Items (OCT 2014)

M.2. Evaluation Factor 1 – Technical Capability

M.3. Evaluation Factor 2 – Past Performance

M.4. Evaluation Factor 3 – Price

M.5. Basis for Award

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 4

SECTION B – SUPPLIES OR SERVICES AND PRICE/COSTS

B.1. Price/Cost Schedule

CLIN Description of Supplies/Services Qty U/I Unit

Price Amount

0001 Benin II Off-Grid Energy Access Project Evaluation

Base Period

Phase 1: Tasks 1 and 2

Period of Performance: TBD

Estimated 8 months

1 LO ______________ ______________

1001 Benin II Off-Grid Energy Access Project Evaluation

Option Period I

Phase 2: Task 3-6

Estimated 10 months

2001 Benin II Off-Grid Energy Access Project Evaluation

Option Period II

Phase 3: Task 7

Estimated 24 months

3001 Benin II Off-Grid Energy Access Project Evaluation

Option Period III

Phase 4: Task 8-11

Estimated 12 months

TOTAL ESTIMATED PRICE: _______________

B.2. FAR 52.216-1 -- Type of Contract (APR 1984)

The Government contemplates award of a time-and-materials (T&M) contract resulting from this solicitation.

(End of Provision)

B.3. Accounting and Appropriation Data

[RESERVED]

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 5

B.4. Funding Type

This contract will be funded, at time of award, with no-year Due Diligence funding. As such, funding obligated to this contract may be used during the life of the contract.

B.5. Limitation of Funds – Incrementally Funded Contracts

In accordance with clause MCC 52.232-72, “Limitation of Funds – Incrementally Funded Contracts,” funds in the amount of [TBD] have been allotted to this contract. It is contemplated that funds now obligated under this contract will cover the work to be performed until [TBD]. If the contractor considers the funds obligated under this contract to be insufficient to cover the work to be performed until the date specified herein, or another date agreed to by the parties, the contractor shall notify the contracting officer in writing and indicate the date on which it expects expended funds to approximate 75 percent of the total amount obligated. The notice shall state the estimated amount of additional funds required to continue performance through the date for which incremental funds are provided.

The Contractor shall not perform work on the contract which exceeds the total amount actually allotted by the

Government to the contract. The Government is not obligated to reimburse the Contractor for costs incurred in excess of the total amount allotted by the Government to this contract. The contractor assumes the risk for any increased costs beyond what the Government obligates.

B.6. Labor Rates & Other Direct Costs (ODCs)

Labor rates at the inception of a contract period will carry through as “fixed-price” without adjustment for the life of that contract period.

Hours by labor category and pricing under ODCs are estimates only and may be increased/decreased as necessary for successful completion of the contract requirements and as approved by the COR/PM. However, the contractor’s expenditures at any given point shall not exceed the total obligated amount or the total ceiling price for this contract as specified below.

Using the template below, Offerors must include all proposed labor categories, fully-burdened hourly rates and Other

Direct Costs (ODCs) for each contract period identified in B.1.

Pricing Summary Per Contract Period

Base

Phase 1

Option I

Phase 2

Option II

Phase 3

Option III

Phase 4

TOTAL

Total Labor Hours

Total Labor Costs $ - $ - $ - $ - $ -

Total ODCs $ - $ - $ - $ - $ -

Total Est. Cost $ - $ - $ - $ - $ -

Base Period (Phase 1) – Period of Performance TBD (est. 8 months)

Item # Description(2) U/I Qty Rate Total

LABOR

0001 hr.

hr.

hr.

hr.

Total Labor Hours - -

Labor Sub-Total

ODCs(1)

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 6

ODC Sub-Total

Total Estimated Value (Base Period)

Option Period I (Phase 2) – Period of Performance TBD (est. 10 months)

LABOR

1001 hr.

hr.

hr.

hr.

Total Labor Hours - -

Labor Sub-Total

1001 Data Collection Subcontracting Costs $400,000.00

Total Estimated Value (Option Period I)

Option Period II (Phase 3) – Period of Performance TBD (est. 24 months)

LABOR

2001 hr.

hr.

hr.

hr.

Total Labor Hours - -

Labor Sub-Total

Total Estimated Value (Option Period II)

Option Period III (Phase 4) – Period of Performance TBD (est. 12 months)

LABOR

3001 hr.

hr.

hr.

hr.

Total Labor Hours - -

Labor Sub-Total

3001 Data Collection Subcontracting Costs $400,000.00

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 7

Total Estimated Value (Option Period III)

TOTAL COST SUMMARY

Description(2) U/I Qty Rate Total

LABOR

hr.

hr.

hr.

hr.

Total Labor Hours - -

Labor Sub-Total

Data Collection Subcontracting Costs $800,000.00

Total Estimated Cost

NOTE (1): MCC anticipates approximately $400,000.00 (Option Period 1) and approximately

$400,000.00 (Option Period 3) worth of data collection subcontracting costs for which the requirements are unknown at this time. To be fair to all vendors providing a quotation, a plug number is utilized for data collection subcontracting costs in the respective years in which data collection is expected to occur. However, if you believe this plug number is inadequate, please advise during the Question and Answer period of the solicitation. “Data Collection

Subcontract(s),” as used in the tables above, refers only to the cost of subcontracting a data collection firm and any associated indirect costs (e.g., subcontract handling fees). It does not include any prime contractor labor associated with overseeing/managing the data collection effort;

this time should be incorporated into the level of effort proposed under the respective Labor CLIN.

Subcontracting costs for data collection are subject to the requirements of FAR 52.244-2, Subcontracts.

NOTE (2): A job description shall be included for each labor category proposed. The job description must identify essential and non-essential tasks that are assigned to a specific position.

It must also identify reporting relationships and shall also describe required qualifications, minimum requirements, working conditions and desirable qualifications.

(End of SECTION B)

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 8

SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK

C.1. Introduction and Background

The Millennium Challenge Corporation (MCC) is a U.S. Government corporation whose mission is to provide assistance that will support economic growth and poverty reduction in carefully selected countries that demonstrate a commitment to just and democratic governance, economic freedom, and investments in their citizenry. For more information on the MCC, please visit www.mcc.gov. The MCC’s Monitoring and Evaluation (M&E) Policy can accessed here:

https://www.mcc.gov/resources/doc/policy-for-monitoring-and-evaluation

The Benin II Compact between the MCC and the Government of Benin was signed on September 9, 2015. The Compact objective is to expand business production and productivity, generate greater economic opportunities for households and improve the capacity to provide public and social services by improving the quantity and quality of the supply of electricity in Benin. The Benin II Compact (the “Program”) entered into force (EIF) on June 22, 2017. A full copy of the Program can be accessed on MCC’s website here: https://www.mcc.gov/where-we-work/program/benin-power-compact. The Program’s

Monitoring and Evaluation Plan is available here: https://www.mcc.gov/where-we-work/m-and-e/benin-power-compact.

The Program targets poor electricity infrastructure, a binding constraint to economic growth in Benin, where the insufficient quantity and quality of electricity results in reduced productivity, output, and investment for businesses, less effective delivery of public and social services and diminished well-being and economic opportunity for households. The root causes of these problems are lack of resources, and public policies and institutions that have been unable to deliver sufficient power to meet growing national demand.

The Program will address these core problems through policy reforms and institutional strengthening, large-scale investments in energy generation and distribution infrastructure, and off-grid electrification to expand access in a country where only one-third of the population has access to electricity. As a result of these interventions, the Program is expected to expand business production and productivity, increase economic opportunities for households, and improve the capacity to provide public and social services.

The Compact includes four Projects, each with a related project objective:

The Policy Reform and Institutional Strengthening Project, with the objective of strengthening the capacity of the

Société Béninoise d’Energie Electrique (“SBEE”), Autorité de Régulation de l’Electricité (“ARE”), and other

Government of Benin institutions to improve the governance, management, and operations of the electricity sector.

It is comprised of the following activities: Policy, Regulation, and Institutional Support; Utility Strengthening; and, Public Information and Education.

The Electricity Generation Project, with the objectives of: (A) increasing the hours of operation for businesses and public and social services; (B) reducing reliance on costlier sources of energy; (C) reducing losses of products and perishable goods; and (D) improving productivity for users of electricity. It is comprised of the Photovoltaic

Generation and Hydroelectric Generation activities.

The Electricity Distribution Project, with the objectives of: (A) increasing the hours of operation for businesses and public and social services, (B) reducing reliance on costlier sources of energy, (C) reducing losses of products and perishable goods, and (D) improving productivity for users of electricity. It is comprised of the Regional Grid

Strengthening, Cotonou Grid Strengthening, and National Electricity Dispatch activities.

http://www.mcc.gov/ https://www.mcc.gov/resources/doc/policy-for-monitoring-and-evaluation https://www.mcc.gov/where-we-work/program/benin-power-compact https://www.mcc.gov/where-we-work/m-and-e/benin-power-compact

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 9

The Off-Grid Electricity Access Project (OGEAP) ($46 million), with the objective of increasing access to electricity and thereby (A) increasing the hours of operation for businesses and public and social services, (B) reducing reliance on costlier sources of energy, (C) reducing losses of products and perishable goods, and (D) improving productivity for users of electricity. It is comprised of the Enabling Environment for Off-Grid Electricity and Off-Grid Clean Energy Facility (OCEF) activities. As they are the focus of this evaluation, these two activities are described in further detail below.

Under the Enabling Environment for Off-Grid Electricity Activity ($5.7 million), MCC is supporting the:

Design and implementation of a national off-grid electrification framework in form and substance satisfactory to

MCC (“National Framework”). The National Framework will articulate a model for off-grid electrification to include regulatory and institutional framework, licensing, tariff evaluations, regulations and technical standards.

Such model will be designed to ensure minimum technical specifications, quality of service standards, licensing, pricing and contracting arrangements, and consideration of gender and social inclusion needs and concerns; and

Development of market information, market characterization, outreach to the private sector and sector donors, and community-led engagement on the OCEF (Off-Grid Clean Energy Facility Activity).

The National Framework consists of the an Off-Grid Electrification Policy, the Off-Grid Electrification Master Plan, and the Off-Grid Regulatory Framework, which the Government of Benin is expected to approve during the summer of 2018 and subsequently adopt.

The Off-Grid Clean Energy Facility Activity ($40.3 million) will increase access to electricity for the currently unconnected majority of the population in rural and peri-urban areas by reducing or removing initial cost and investment barriers for off-grid electricity service providers. MCC funding will support the establishment of OCEF and grants issued there under in four primary windows:

Critical public infrastructure to provide stand-alone electricity generation capability (“Window One”);

Mini-grids providing electricity generation and distribution for household, commercial, agricultural and industrial use (“Window Two”);

Household generation, storage, and productive uses, such as renewable energy source devices for individual families

(“Window Three”); and

Energy efficiency measures for buildings, facilities and installations (“Window Four”, and together with Window

One, Window Two and Window Three, the “OCEF Windows”).

There are expected to be two rounds of call for proposals for grants ranging between $100,000 and $5,000,000. The OCEF is expected to award no more than 20 grants between the two calls for proposals. Grants are expected to last 24 months.

Grant activities are expected to start immediately after the final grant agreements are signed. Grant activities may be implemented anywhere in Benin. For Window 1, 2, and 3, there are requirements regarding distance from the electricity grid.

The target amount for grant awards in the first round is $12,000,000. The first round will exclude Window Two as the release of funding for this window is subject to satisfaction of the Condition Precedent, which states the Government of

Benin will have implemented and be in compliance with the Off-Grid Policy. The first round was launched on February 22, 2018 with grantees expected to be pre-selected in August 2018. After pre-selection, the grantees will receive technical assistance in areas including environmental and social performance, gender and social inclusion, monitoring and evaluation, and economic analysis before finalizing proposals for approval. Final grant agreements expected to be signed in February

2019. MCC funding for Window Two is conditioned on the Government of Benin’s implementation and adoption of the

National Framework.

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 10

The second round of call for proposals for windows two and four only is expected to be launched in October 2018 with grantees expected to be pre-selected in March 2019 and final grant agreements expected to be signed in July 2019. The target amount for grant awards in the second round is $20,000,000. MCC may provide the contractor with a draft work plan upon being awarded this contract.

More information on OCEF including the submission guideline for grantees can be found online: https://ocef.bj/en/

The below program logic outlines the anticipated logic and theory of change for the Program.

Compact Program Logic

Electricity Generation ProjectElectricity Distribution Project

Improved Voltage

Quality and Stability for

Users

Greater Hours of

Operation for

Businesses and Public

Services

Lower Reliance on

Costlier Energy Sources

(Diesel Generators, Kerosene, etc)

Reduced Number and

Duration of Outages

Reduced Losses of

Products and Perishable

Goods from Outages

Reduced Degradation of

Equipment Due to

Unstable Electricity

Economic Growth and Poverty Reduction via:

Expanded Business Production and Productivity

Greater Economic Opportunities for Households

Improved Capacity to Provide Public and Social Services

Reduced Technical

Losses and Improved

Distribution Capacity

Reduced Commercial

Losses

SBEE Is More

Financially Sustainable

Increased Capital for

Maintenance and New

Capital Investments

Increased Domestic

Generation Output

Increased Access to

Electricity

Improved Productivity for Beneficiaries

Improved Management, Operations, and

Maintenance within

SBEE

Improved Governance of the Energy Sector

Policy Reform and Institutional

Strengthening Project

Off-Grid Electricity Access

Project

Avoided Demand

Among Energy Users https://ocef.bj/en/

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 11

The below program logic outlines the anticipated logic and theory of change for Off-Grid Electricity Access Project

Off-Grid Electricity Access Project Program Logic

Enabling Environment for Off-Grid Electricity

Activity

(legal and institutional framework, procedures); Institutional

Support to ANADER, ABERME, and the Regulator, Enabling

Environment for Solar)

DE2

Increased Investment in

Off-Grid Energy

Solutions

DE1

Increased Access to

Electricity

Off-grid Clean Energy

Activity

Policy, Strategy and

Master Plan Regulatory

Framework approved by the GoB and

Promolagated

Increased Employment

Opportunites

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 12

The OCEF theory of change is included below:

Source: Grant Facility Manager (Adapted and Translated by MCC)

C.2. Scope

Provide Independent Evaluation Services in support of the Off-Grid Electricity Access Project (OGEAP). For the avoidance of doubt, this includes both activities under this Project: the Enabling Environment for Off-Grid Electricity Activity and the

Off-Grid Clean Energy Facility (OCEF) Activity. The Contractor will also evaluate the grants under the OCEF Activity, but not all grants must be evaluated.

C.3. Evaluation Questions and Guidance on Methodology

A. Evaluation Questions

The anticipated research questions are listed below. The Contractor will be expected to work with MCC, the MCA, and local stakeholders to assess and finalize this list as a part of the elaboration of the Evaluation Design Report. To the extent possible, these questions shall also include an assessment of why or why not changes were observed, a discussion of major unanticipated outcomes (positive or negative), whether the observed changes are likely to be sustainable, as well as an analysis as to how (and why) effects may vary by gender. MCC’s priority in measuring attributable impacts and assessing cost-effectiveness inform the evaluation questions included below. The Contractor should use the MCC M&E Policy as a reference for its proposed revisions to these questions.

Outcomes of the Off-Grid Clean Energy Facility (OCEF) Activity

Were the selection criteria and procedures defined in the manual well aligned to the objectives of selecting and scaling effective, sustainable models of off-grid energy services and expanding access to energy to underserved communities?

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 13

To what extent did the grant facility’s evaluation criteria incentivize the proposal and acceptance of grants that best advanced its goals?

To what extent did the evaluation criteria incentivize all applicants to reveal the minimum subsidy required for their investment to move forward? Was the cost sharing amount sufficient to allocate the risks appropriately to the grantee?

To what extent did the grant facility generate information externalities that encouraged subsequent investment in the off-grid clean energy sector? Did the grant facility catalyze additional commercial financing of the off-grid energy sector in Benin?

To what extent did the grant facility resolve discovery and/or coordination market failures limiting investment in

Benin’s off-grid clean energy sector?

Did the grant facility demonstrate scalable business models that would be sustainable in the absence of subsidies? To what extent did the facility encourage coordinated, complementary investments that increased the viability/sustainability of facility investments? To what extent are the grantees’ business models sustainable?

Implementation of the Off-Grid Clean Energy Facility (OCEF) Activity

Did the Project Selection Panel and Investment Committee select grants and the grant facility manager oversee grants in accordance with the OCEF manual?

To what extent was the technical assistance provided to grantees relevant to the gaps identified in the grantee proposals and to the grantee’s objectives? Did grantees perceive the technical assistance they received as hindering or enabling their ability to meet the grant’s objectives?

To what extent were OCEF grants implemented in accordance to the grantees’ implementation plans?

Outcomes of the Enabling Environment for Off-Grid Electricity Activity

To what extent has the regulatory framework for off-grid electricity been implemented in Benin? How do organizations interested in the off-grid electricity market in Benin perceive the framework?

What are private sector actors’ experiences in attempting to enter or grow in the off-grid clean energy market? Did the presence of an off-grid regulatory framework make private sector actors more or less likely to enter the off-grid clean energy market in Benin?

Grant Outcomes (aligned with the Project Objective)

Did the Project increase the hours of operation of businesses and public services?

Did the Project reduce the reliance on costlier energy sources?

Did the Project reduce losses of products and perishable goods?

Did the Project improve the productivity for beneficiaries? (These research question are in bold, because they correspond directly to the Project Objective).

Did the Project impact the adoption of electricity-consuming technologies? Which technologies and to what extent are they being used? What are the characteristics of the individuals, households, and/or businesses that adopted these technologies? What behaviors and practices may have changed because of technology adoption?

For which type of enterprises did electrification stimulate investment? To what extent did electrification stimulate investment in (1) non-tradable services (e.g. restaurants, hairdressers, etc), (2) agricultural production, (3) and other tradable products and services?

How many beneficiaries received a basic level of electricity through the OCEF grants? How does that result compare with the national utility’s targets and achievements during the same period? What enabled or hindered the grantees’ ability to increase access to electricity?

How does OCEF’s cost of increasing electricity access and consumption compare with other projects in Benin? Were the grants more or less cost-effective than these other projects?

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 14

Did Window Four of OCEF increase the efficient use of energy among targeted beneficiaries?

B. Guidance on Methodology

MCC procures two different types of evaluations: Impact Evaluations and Performance Evaluations.1

The possibility of including an impact evaluation component should not be ruled out. However, there are several factors limiting the possibility of an impact evaluation including, but not limited to:

- Random assignment to the treatment is unlikely to occur

- The OCEF Activity will consist of differing sub-activities making it impossible to define a single treatment

- The variety of anticipated outcomes among the four windows, especially Windows One and Four.

The Contractor’s methodology shall cover the various levels of the evaluation reflected in the categories of evaluation questions above. Those categories are subject to change based on the Contractor’s proposed approach.

There are inherent challenges in evaluating grant facilities. Projects implemented through grant facilities often lack the key evaluability elements such as an evidence-based problem analysis, the logic or theory of change, the performance indicators, the geographic location and beneficiaries, and the risks and assumptions. If these are defined, it usually occurs when a final grant is awarded. Implementation often follows soon after grant award resulting in a short period of time for an evaluator to collect baseline data. Furthermore, the variety across projects of these elements makes it difficult to adopt a single evaluation approach for all or several grants. It also creates a larger burden for data collection and analysis.

The Contractor and MCC will make a final determination on the number of grants that the analysis will cover at the

Evaluation Design Report phase. The Evaluation should cover at least one grant per window. For each grant that is evaluated, the evaluator should estimate the grant’s evaluation-based economic rate of return (ERR). The evaluator should estimate, to the extent possible, the aggregate ERR for each grant window, and for the overall Project.

The Contractor’s methodology shall include a single quantitative data collection effort for multiple grants under both rounds that covers the Project Objectives and is sufficient for estimating the ERR. The rules of implementation will determine how many grants the survey will cover. The respondents may include households, enterprises, and public and social services.

The Window Two and Three grants are expected to have similar enough project logics that covering most or all of the grants under such a survey may be feasible. Additional quantitative data collection sufficient for estimating the grants’ ERR is required for at least one grant under both Window One and Four.

C.4. Roles and Responsibilities

A. Independence:

MCC’s Policy on Independent Evaluations. Based on the 1991 DAC Principles for Evaluation of Development Assistance, MCC’s Evaluation Policy requires independent evaluations of each MCC project. MCC’s implementation of this independence policy adheres to these principles, especially the following: “Impartiality and independence will best be

1 MCC defines an impact evaluation as a study that measures the changes in income and/or other aspects of well-being that are attributable to a defined intervention. Impact evaluations require a credible and rigorously defined counterfactual, which estimates what would have happened to the beneficiaries absent the project. MCC defines a performance evaluation as a study that seeks to answer descriptive questions, such as: what were the objectives of a particular project or program; what the project or program has achieved; how it has been implemented; how it is perceived and valued;

whether expected results are occurring and are sustainable; and other questions that are pertinent to program design, management, and operational decision making. MCC’s performance evaluations also address questions of program impact and cost-effectiveness http://www.oecd.org/dac/evaluation/dcdndep/45438179.pdf

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 15 achieved by separating the evaluation function from the line management responsible for planning and managing development assistance.” (P.6.)

Organizational Conflicts of Interest. MCC-M&E is responsible for adhering to our policy-driven requirements for independence. These requirements are distinct from Organizational Conflicts of Interest (OCI) proscribed in the Federal

Acquisition Regulations. Interpretations and enforcement of the OCI statutes are the remit of our Office of General Counsel.

Independence of Evaluation Team. Evaluation team members must be impartial and therefore should not have contributed to the design or implementation of the project being evaluated. In cases where individuals or firms that have worked on the project are proposed to be part of the evaluation team, a justification and independence risk mitigation plan must be provided to MCC for consideration.

The Independent Evaluator’s Role. To remain consistent with the principles of independent evaluations during project planning and implementation, the independent evaluator’s feedback is restricted to reporting evidence and findings.

Recommendations shall not be made before completing a final evaluation as they can influence project planning or implementation, thereby jeopardizing the independence an evaluation. Prescriptions and recommendations may be included in final evaluations.

Real-time feedback to project implementation. MCC supports monitoring and evaluation activities intended to provide feedback on how to improve program planning and implementation. But this support shall be separate from the independent evaluator. Any contribution of the Independent Evaluator to this feedback shall be limited to reporting evidence and findings. Any exceptions to this rule must be approved in advance by the EMC.

B. MCC & MCA Responsibilities: The MCC, MCA, along with the Independent Evaluator, play critical roles in designing, implementing, and disseminating the independent evaluations. The MCC and MCA roles are as follows (For further detail see Section J: Evaluation Management and Review Process):

1. Millennium Challenge Corporation (MCC): MCC is responsible for oversight of the Independent Evaluator and quality control of evaluation activities, including the following specific responsibilities:

a. Assess when a program is ready for evaluation planning through evaluability assessment;

b. Determine what program components (Project(s), Activities, Sub-Activities) will be covered by the evaluation;

c. Set the evaluation questions to achieve accountability and learning objectives;

d. Build buy-in and ownership of the evaluation;

e. Contract and Supervise the Independent Evaluator;

f. Conduct quality reviews of all evaluation products (reports, questionnaires, etc);

g. Facilitate public dissemination efforts to inform decision-makers on learning generated by the evaluation;

h. Oversee preparation of public-use data that ensures appropriate balance of enabling verification of analysis and broad use of the data with adherence to promises of confidentiality to survey respondents; and

i. Determine the evaluation budgets.

2. Millennium Challenge Account (MCA) and country partners: MCA and country partners are responsible for building local ownership and commitment to the evaluation, and quality control of evaluation activities, including the following specific responsibilities:

a. Assess when a program is ready for evaluation planning through evaluability assessment;

b. Set the evaluation questions to achieve accountability and learning objectives;

c. Build buy-in and ownership of the evaluation;

d. Conduct quality reviews of all evaluation products (reports, questionnaires, etc); and

e. Facilitate public dissemination efforts.

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 16

C.5. Tasks

Independent Evaluator (hereafter, the Contractor): The Contractor is responsible for the overall design, implementation, and dissemination of the evaluation, including the following specific responsibilities:

a. Develop the most rigorous evaluation design feasible given rules of program implementation (Tasks 1-2)

b. Support MCC and MCA to build buy-in and ownership of evaluation. (Task 2 and Task 7)

c. Develop evaluation materials that are held to international standards (Task 3 and Task 8)

d. Ensure Institutional Review Board clearance of survey protocols (Task 4 and Task 9)

e. Manage the data collection firms (Task 4 and Task 9)

f. Ensure data quality during collection and entry through supervisions and management (Task 4 and Task

9)

g. Lead data cleaning, analysis, and interpretation of results (Task 5 and Task 10)

h. Produce high quality, credible, and transparent evaluation reports (Task 5 and Task 10)

i. Lead preparation of public-use data that ensures appropriate balance of enabling verification of analysis and broad use of the data with adherence to promises of confidentiality to survey respondents (Task 6 and

10)

j. Lead public dissemination efforts (Task 11)

The following section details the required sub-tasks for each evaluation task.

Task 1: Assess Evaluation Plan

The Contractor shall be responsible for conducting an Evaluability Assessment on all existing documentation related to the program and proposed evaluation plan provided by MCC and MCA using the MCC Evaluability

Assessment Tool (See Section J) as a guide. The Evaluability Assessment shall cover all grants and use the facility manager’s evaluability assessments as a reference. The Evaluability Assessment Report summarizes the overall evaluability of the Project.

The Contractor shall produce an Evaluability Assessment Report that provides an independent assessment of the program being evaluated. This assessment should be guided by the Evaluability Assessment Tool and should assess the quality, completeness, and/or appropriateness of the Problem Diagnostic, Program Logic, Cost-Benefit

Analysis, Risks & Assumptions, Monitoring Indicators, and Proposed Evaluation Questions. The report should aim to convey the Contractor’s overarching assessment of the coherence of the program design, the underlying economic analysis, and the M&E approach.

MCC and the Contractor should identify all MCC and MCA self-evaluation activities and determine how the independent evaluation will (i) validate findings of self-evaluation(s), (ii) use self-evaluation(s) to inform the independent evaluation, particularly around understanding what was designed, what was implemented, and understanding any necessary course corrections, (iii) complement the self-evaluation(s) by expanding on self-evaluation activities.

Develop a Work Plan with expected deadlines for deliverables. Any revisions to the Work Plan shall be submitted in the Evaluator’s Progress Reports.

Task 2: Develop Evaluation Design Report

The Contractor shall conduct missions to the field as needed to discuss program design and implementation issues to inform the evaluation design, as well as engage local stakeholders (MCA and Implementing Entities, as well as local government) on the proposed evaluation. This includes hosting local stakeholder workshops to develop/refine research questions, assess appropriate and feasible quantitative and qualitative evaluation methodologies, develop evaluation timeline, and build local stakeholder commitment to the evaluation design.

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 17

The Contractor shall develop the Evaluation Design Report, including all technical analysis required for assessing and confirming (as necessary) internal validity, power calculations, and required sample sizes. The Evaluation

Design Report outline shall be developed in consultation with the MCC M&E lead. Please reference the Evaluation

Design Report Outline (See Section J) as a guide. MCC expects a single evaluation design report that describes multiple designs to reflect the different levels of the evaluation research questions and the possibility of separate grantee-level evaluations.

As per MCC’s Gender Policy and Gender Integration Guidelines, the Contractor shall ensure that evaluation design, data collection requirements, and analysis address gender and other social differences as appropriate. This involves including research questions that address social and gender dimensions of the project logic, assumptions, and project design; disaggregating impacts by gender, age, income, and other social differences (e.g. ethnicity) as appropriate;

including special modules or studies as needed.

The Contractor shall estimate the project’s post-compact economic rate of return (ERR):

o In the Evaluation Design Report, the Evaluator will specify which key input, output and outcome parameters modeled in the MCC CBA will be measured as part of the proposed evaluation design. At this stage, it is likely that only the CBA models for windows two and three will be available. MCC expects to determine a CBA model for Window Four after the first projects are designed. The CBA models for

Window One will likely be done on a case-by-case basis as information on project design becomes available. If the Evaluator believes it is infeasible or prohibitively expensive to measure a benefit stream in the MCC CBA, this should be made explicit and a justification provided for MCC to consider. The

Contractor shall pan to revise the CBA methodology section of the Evaluation Design Report when it had received MCC’s CBA models for Window One and Window Four.

o The Contractor should propose a CBA methodology to estimate the project’s post-compact ERR. The methodology may be based on an existing MCC model or developed by the Evaluator. It should be documented in the EDR, citing sources where relevant. The Evaluator may recommend to MCC that it is not feasible or worthwhile to build a CBA model of the project. A justification for this recommendation should be provided in the EDR for MCC’s consideration.

The Contractor shall document feedback from local stakeholders and the response to that feedback in an annex to the final report. In order to obtain feedback, the Contractor may lead local workshops and/or virtual reviews of the

Design Report by MCA and Implementing Entities.

o In accordance with Section H, the Contractor is responsible for allowing any key local stakeholders the opportunity to submit a public Statement of Difference or Support, as agreed with MCC and the

MCA/designated representative.

The Contractor shall submit the Evaluation Design Report to MCC and may be asked to present to the Evaluation

Management Committee (See Section J). All feedback and response to feedback shall be documented by the

Contractor in an annex to the final report.

o In accordance with Section H, MCC may: (i) select to obtain external peer review; and (ii) make the Report and peer reviews public on the MCC website.

The Contractor shall produce a Nesstar Metadata file (See Section J) summarizing the final Evaluation Design

Report for dissemination on the MCC Evaluation Catalog. Prior to publication, the Contractor should review the current version of MCC’s Evaluation Microdata Guidelines (See Section J) for guidance on preparation of public documentation that links to microdata to reduce re-identification risk.

The Contractor shall continue to maintain the Evaluation Design Report as the evaluation develops. The Evaluation

Design Report will remain a living document that is updated by the Contractor as necessary and resubmitted to

MCC as necessary.

Task 3: Develop Evaluation Materials

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 18

The Contractor shall develop data collection firm TORs and procure the services of a data collection firm(s).

The Contractor shall lead development of any necessary survey instruments for primary data collection (including questionnaires and accompanying enumerator manuals) or a data extraction approach for existing sources. These shall be aligned with the data requirements of the CBA models.

The Contractor will develop survey materials in English and French, unless otherwise agreed with MCC. It is possible that survey instruments will require translation into national languages of Benin, such as Fon, Yoruba, or

Fulfulde. When necessary, the Contractor will utilize existing instruments to improve coordination across MCC independent evaluations for meta-analysis. The Contractor is responsible for pretesting2 survey instruments during their development and reviewing translations and back-translations conducted by data collection firms prior to data collection start (as necessary).

The Contractor is responsible for ensuring household surveys last no longer than 120 minutes without written consent from the Evaluation Management Committee.

The Contractor shall document feedback from local stakeholders and response to the feedback. In order to obtain feedback, the Contractor may lead local workshops and/or virtual reviews of the Evaluation Materials by MCA and

Implementing Entities.

The Contractor shall submit the Evaluation Materials to MCC and may be asked to present to the Evaluation

Management Committee. All feedback and response to feedback will be documented by the Contractor.

o The Contractor is responsible for the quality of the English and local language survey instruments.

Task 4: Undertake baseline data collection

The Contractor shall be responsible for developing and overseeing all data collection and entry protocols and implementation. This requires reviewing the current version of MCC’s Evaluation Microdata Guidelines (See

Section J) and ensuring appropriate Institutional Review Board (IRB) review and approval of research protocols, survey instruments, and informed consent procedures (see MCC Informed Consent Statement Template (See

Section J)).

o In accordance with Section H, the Contractor should assess the local IRB requirements and determine whether or not additional, international-level IRB approval is required in order to ensure adherence to international standards.

If there are local requirements, the Contractor must abide by them.

If the local IRB is HHS registered, no additional IRB is required.

If the local IRB is not HHS registered, the Contractor also needs to sub-contract an HHS registered

IRB.3

The Contractor shall be responsible for overseeing the training of data collection staff, field work, and data entry.

The Contractor is responsible for pilot testing the survey instruments and field procedures as a part of enumerator training. This must be managed in coordination with MCC and MCA. The Contractor is responsible for reporting on the average survey time resulting from the pilot test and from field work.

The Contractor shall be responsible for conducting and documenting quality control checks of data as it is collected and entered in order to reduce data collection and entry error.

The Contractor shall be responsible for establishing a system for frequent communication with the data collection firm(s) throughout data collection and entry, through email, phone, and site visits.

2 Note that MCC differentiates between the “pre-test”, defined as the testing of the instrument for content and flow, versus the “enumerator pilot”, an exercise completed during training for enumerators to practice with the instrument, and for the survey team to practice data flow, but not to result in major changes to the instrument.

3 HHS registered IRBs are found here - https://ohrp.cit.nih.gov/search/search.aspx?styp=bsc

95332418Q0109 (Benin II Off-Grid Energy Access Project Evaluation) Page 19

Task 5: Develop Baseline Report and Data Documentation Package

The Contractor shall review raw files from the survey firm and provide any critical feedback on production of the survey firm’s Data Collection Report.

Before conducting baseline analysis, the Contractor shall review the current version of MCC’s Evaluation

Microdata Guidelines (See Section J) for guidelines on management, documentation, and dissemination of the microdata.

The Contractor shall be responsible for developing the Baseline Report. The Baseline Report outline shall be developed in consultation with the MCC M&E lead. Please reference the Baseline Report Outline as a guide (See

Section J).

The Contractor shall ensure baseline analysis including gender and social analysis is part of the baseline report as described in the evaluation design report (task 2), including examining differences in baseline characteristics by gender, age, income and other social variables as appropriate.

The Contractor shall document feedback from local stakeholders and response to feedback in an annex to the final report.