MCC-15-RFQ-0095_Bamako_Airport_Performance_Evaluation.pdf

PAGE 1 OF 1. REQUISITION NO.

2. CONTRACT NO. 3. AWARD/EFFECTIVE DATE 4. ORDER NO. 5. SOLICITATION NUMBER 6. SOLICITATION ISSUE DATE

a. NAME b. TELEPHONE NO. (No Collect Calls) 8. OFFER DUE DATE/LOCAL

TIME

9. ISSUED BY CODE 10. THIS ACQUISITION IS UNRESTRICTED OR SET ASIDE: % FOR:

SMALL BUSINESS

HUBZONE SMALL

BUSINESS

SERVICE-DISABLED

VETERAN-OWNED

SMALL BUSINESS

WOMEN-OWNED SMALL BUSINESS

(WOSB) ELIGIBLE UNDER THE WOMEN-OWNED

SMALL BUSINESS PROGRAM

EDWOSB

8(A)

NAICS:

SIZE STANDARD:

11. DELIVERY FOR FOB DESTINA-

TION UNLESS BLOCK IS

MARKED

SEE SCHEDULE

12. DISCOUNT TERMS

13a. THIS CONTRACT IS A

RATED ORDER UNDER

DPAS (15 CFR 700)

13b. RATING

14. METHOD OF SOLICITATION

RFQ IFB RFP

15. DELIVER TO CODE 16. ADMINISTERED BY CODE

17a. CONTRACTOR/OFFEROR CODE FACILITY CODE 18a. PAYMENT WILL BE MADE BY CODE

TELEPHONE NO. DUNS: DUNS+4:

PHONE: FAX:

17b. CHECK IF REMITTANCE IS DIFFERENT AND PUT SUCH ADDRESS IN OFFER 18b. SUBMIT INVOICES TO ADDRESS SHOWN IN BLOCK 18a UNLESS BLOCK BELOW IS CHECKED

SEE ADDENDUM

19. 20. 21. 22. 23. 24.

ITEM NO. SCHEDULE OF SUPPLIES/SERVICES QUANTITY UNIT UNIT PRICE AMOUNT

(Use Reverse and/or Attach Additional Sheets as Necessary)

25. ACCOUNTING AND APPROPRIATION DATA 26. TOTAL AWARD AMOUNT (For Govt. Use Only)

27a. SOLICITATION INCORPORATES BY REFERENCE FAR 52.212-1, 52.212-4. FAR 52.212-3 AND 52.212-5 ARE ATTACHED. ADDENDA ARE ARE NOT ATTACHED.

27b. CONTRACT/PURCHASE ORDER INCORPORATES BY REFERENCE FAR 52.212-4. FAR 52.212-5 IS ATTACHED. ADDENDA ARE ARE NOT ATTACHED

28. CONTRACTOR IS REQUIRED TO SIGN THIS DOCUMENT AND RETURN _______________ 29. AWARD OF CONTRACT: REF. ___________________________________ OFFER COPIES TO ISSUING OFFICE. CONTRACTOR AGREES TO FURNISH AND DATED ________________________________. YOUR OFFER ON SOLICITATION DELIVER ALL ITEMS SET FORTH OR OTHERWISE IDENTIFIED ABOVE AND ON ANY (BLOCK 5), INCLUDING ANY ADDITIONS OR CHANGES WHICH ARE ADDITIONAL SHEETS SUBJECT TO THE TERMS AND CONDITIONS SPECIFIED SET FORTH HEREIN IS ACCEPTED AS TO ITEMS:

30a. SIGNATURE OF OFFEROR/CONTRACTOR 31a. UNITED STATES OF AMERICA (SIGNATURE OF CONTRACTING OFFICER)

30b. NAME AND TITLE OF SIGNER (TYPE OR PRINT) 30c. DATE SIGNED 31b. NAME OF CONTRACTING OFFICER (TYPE OR PRINT) 31c. DATE SIGNED

AUTHORIZED FOR LOCAL REPRODUCTION (REV. 2/2012)

PREVIOUS EDITION IS NOT USABLE Prescribed by GSA - FAR (48 CFR) 53.212

7. FOR SOLICITATION

INFORMATION CALL:

STANDARD FORM 1449

OFFEROR TO COMPLETE BLOCKS 12, 17, 23, 24, & 30

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL ITEMS 64

MCC-15-RFQ-0095 8-14-2015

David Fuller fullerdf@mcc.gov 8-21-2015 Noon Eastern

Millennium Challenge Corporation Contracts and Grants Management Division 875 15th St, NW Suite 200 Washington DC 20005

X 0

541690

N/A

X

N/A

X

CGM

875 Fifteenth St., NW

Contracts and Grants Management Division 875 15th St., NW Suite 200

Interior Business Center M/S D-2773 7301 West Mansfield Avenue Lakewood CO 80235-2230

MCC DPE/M&E Request for Quotation

Independent Evaluation Services

Bamako Airport Performance Evaluation - Mali

See CONTINUATION Page

X X

MCC-15-RFQ-0095 Page 2 of 64 Performance Evaluation Mali Bamako Airport Project

SUPPLIES OR SERVICES AND PRICES/COSTS

Schedule Accounting and Appropriation Data Limitation of Funds – Incrementally Funded Contracts Time and Materials No-Year Funding

DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK

Scope Background and Objectives Roles and Responsibilities:

Roles and Responsibilities Tasks

PACKAGING AND MARKING

Packaging and Marking

INSPECTION AND ACCEPTANCE

DELIVERIES OR PERFORMANCE

Deliverables Reporting Travel Requirements Deliverable Instructions Acceptance Criteria Period of Performance Place of Performance

CONTRACT ADMINISTRATION DATA

SPECIAL CONTRACT REQUIREMENTS

MCC 52.232-72 Limitation of Funds – Incrementally Funded Contracts (NOV 2006) MCC 52.243-70 Increase In Services (OCT 2006) Key Personnel Provision on Peer Review and Posting of Reports Provision on Insurance Maintenance Disclaimer for Deliverables Provision on Stakeholder Input for Evaluation Products Provision on Human Subjects Protection Personally Identifiable Information and Confidentiality Provision on Delivered Work Product Free From 3rd Party Rights Special Contract Requirements

CONTRACT CLAUSES

FAR 52.217-8 Option to Extend Services (NOV 1999) FAR 52.217-9 Option to Extend the Term of the Contract (MAR 2000) FAR 52.232-18 Availability of Funds (APR 1984) Contract Clauses

LIST OF ATTACHMENTS

Annex 1 MCC Independent Evaluation Design Report Outline Template Annex 2 Informed Consent Template Annex 3 MCC Data Documentation and Anonymiaztion Guidelines Annex 4 Nesstar Template Annex 5 Baseline Report Template Annex 6 Interim/Final Report Template Annex 7 TDY SOW Template

REPRESENTATIONS, CERTIFICATIONS, AND OTHER STATEMENTS OF OFFERORS OR

RESPONDENTS

INSTRUCTIONS, CONDITIONS, AND NOTICES TO OFFERORS OR RESPONDENTS

FAR 52.216-1 Type of Contract Inquiries / Questions Concerning the Solicitation Quotation Due Date Quotation Volumes Quotation Format Page Limits Preparation of Volume 1 – Technical Quotation Preparation of Volume II – Cost Quotation Staffing and Required Expertise

MCC-15-RFQ-0095 Page 3 of 64

Offeror Interview

EVALUATION FACTORS FOR AWARD

FAR 52.212-2 Evaluation-Commercial Items (JAN 1999)

MCC-15-RFQ-0095 Page 4 of 64

SUPPLIES OR SERVICES AND PRICES/COSTS

Schedule

CLIN DESCRIPTION OF SUPPLIES/SERVICES QTY UNIT UNIT PRICE AMOUNT

0001 Performance Evaluation - Mali Bamako

Airport Project Time and Materials (T&M) Base Period

1.00 LO $___________ $___________

0002 Data Collection Costs Optional Task Time and Materials (T&M) Plug Number Base Period

1.00 LO $100,000.00 $100,000.00

GRAND TOTAL $___________

Note: MCC anticipates out-year data collection costs for which the requirements are unknown at this time. To be fair to all vendors providing a quotation, a plug number is utilized for CLIN 1002. However, if you believe this plug number is inadequate, please advise during the Question and Answer period of the RFQ. More than one data collection requirement may be required. Once requirement(s) become known, a quotation will be requested from the Contractor and a cost will be negotiated and memorialized in a modification to this call order.

Labor Rates

Labor Rates at the inception of a Phase will carry through without adjustment for the life of that Phase.

Direct Labor Rates

[RESERVED]

Accounting and Appropriation Data

[RESERVED]

MCC-15-RFQ-0095 Page 5 of 64

Limitation of Funds – Incrementally Funded Contracts In accordance with clause MCC52.232-72, “Limitation of Funds – Incrementally Funded Contracts,” funds in the amount of [fill in amount] have been allotted to this contract. It is contemplated that funds now obligated under this contract will cover the work to be performed until [fill in date].

The Contractor shall not perform work on the contract which exceeds the total amount actually allotted by the Government to the contract. The Government is not obligated to reimburse the Contractor for costs incurred in excess of the total amount allotted by the Government to this contract. The contractor assumes the risk for any increased costs beyond what the Government obligates.

Time and Materials No-Year Funding The call order is funded with No-Year Due Diligence funding. As such, funding obligated to this call order may be used during the life of the order.

MCC-15-RFQ-0095 Page 6 of 64

DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK

Scope The Contractor shall provide Independent Evaluation Services in support of the Bamako Airport Performance Evaluation. This scope of work lays out the requirements for a performance evaluation of the MCC activities related to the technical assistance and infrastructure improvements to the Bamako International Airport

Background and Objectives

The primary functions of the contractor are to work with MCC and Malian Stakeholders in:

Articulating more specific research questions to be answered by the evaluation;

Identifying end uses in MCC and Malian decision making for research questions;

Developing an evaluation design for answering evaluation questions;

Implementing agreed upon evaluation design as outlined in the below tasks and deliverables.

This background section first lays out the analysis that was the result of the due diligence that lead to the decision to invest in the Bamako Airport as part of the overall Mali Compact. Of course this analysis was completed before the 2012 military coup and subsequent early termination of the Compact in August, 2012 .

As a result of these events much of the analysis that drove the original investment has become moot. Hence any evaluation of the due diligence analysis that drove the initial investment should be considered on its own and not against these subsequent events. Followed by an overview of that analysis is a description of the various activities carried out under the Airport project.

MCC Due Diligence Analysis

Economic growth and poverty reduction depend on enhanced access to markets and trade, but Mali’s access is severely constrained. The Airport Project sought to expand Mali’s access to markets and trade through improvements to the transportation infrastructure at the Airport, and better management of the national air transport system.

MCC-15-RFQ-0095 Page 7 of 64

Mali is landlocked and depends heavily on inadequate rail and road networks that result in high transportation costs, as well as on freight transport through ports in unstable countries, such as Conakry, Guinea (Bamako’s closest port which is 1000 km away) and Abidjan, Cote d’Ivoire. In the last few years, the instability in Cote d’Ivoire has dramatically limited Mali’s market access.

Before the outbreak of the Ivorian crisis, 70% of Malian exports were leaving via the port of Abidjan. In 2003, this amount dwindled to less than 18%.

Mali cannot control overland routes to international and regional markets.

Therefore, air traffic has become Mali’s lifeline for transportation of both passengers and export products.

Malian exports are predominately agricultural. Rural small-scale producers will likely benefit from increased exports in high-value products such as mangoes, green beans, tomatoes, and gum arabic. Additionally, international tourists arriving at the Airport spend the majority of their time in rural areas, hence benefiting businesses in far-away places such as Timbuktu, Mopti, and Djenné. The improved management of the national airport system will facilitate links to primary destinations through regional air travel. The GoM recognizes the importance of improved air transportation infrastructure.

Mali’s PRSP for 2002 includes rehabilitation of airport infrastructure to “promote access of Malian producers to domestic and international markets.”1

The deteriorating conditions at the Airport could have limited the Airport’s capacity to handle air traffic growth if significant capital improvements are not made. The Airport’s basic infrastructure dates from 1974, is in poor condition, and is inadequate to handle increased passenger and cargo traffic.

On the airside, the runway was too short to accommodate large aircraft without take-off load penalties, the aeronautical pavements urgently need resurfacing and reinforcement, the air navigation aids are reaching the end of their useful life, and airfield security is deficient. On the landside, the passenger terminal building is too small to handle current traffic volumes at acceptable levels of service, and the facilities and equipment are in poor physical condition.

The limitations of the design parameters (geometry and bearing strength) for much of the airside infrastructure elements represent constraints to the potential growth of traffic at the Airport. The pre-existing length of the runway (2700 m) does not permit long-haul take-offs using B747-class aircraft without load penalties and the surface condition and structural strength of the airside pavements (runway, taxiways and apron) do not allow for heavier aircraft. The persistence of this situation reduces the profitability of potential air services and therefore the attractiveness of Mali as a destination and/or trans-shipment point. The air navigation equipment was also reaching the end of its useful life and, although adequate for current operations, and had to be replaced and up-graded to meet the needs of future traffic.

1 GoM Poverty Reduction Strategy Paper 2002.

MCC-15-RFQ-0095 Page 8 of 64

This Project was intended to remove constraints to traffic growth and make the Airport more efficient in both passenger and freight handling through infrastructure airside and landside improvements, as well as the establishment of appropriate institutional mechanisms to ensure effective management, operation, and maintenance of the Airport facilities over the long term.

Description

Forecast Growth in Traffic (Passenger and Cargo)

In 2005, the Airport handled approximately 516,000 arriving and departing passengers and 6,500 tons of air cargo. The Airport has experienced steady passenger growth over the past two decades, indicating a strong potential for growth in demand. Total annual passengers at the Airport grew by an average rate of 5.1% per annum since 1981 and 7.0% over the most recent five-year period. Between 1994 and 2004, Mali experienced the second-highest passenger growth among West African countries after Ghana. For the next twenty years, Mali’s international cargo traffic is projected to increase by 5.7% annually.

C.2.4.1.1. Airside Constraints

If the Airport were to make investments only in the passenger terminal, without the associated investments to the airfield, the Airport would suffer serious long-term negative impact in the ability of airlines to operate within Mali. Over time, without proper improvements to the airfield, the runway would deteriorate to the point where the pavement would be unsafe for heavy aircraft operation. Airlines would thus have to operate with less than maximum loads, resulting in less efficient airline operation. This would translate into lower passenger volumes, higher operating costs, and ultimately higher ticket prices to the traveler.

Starting in 2011, the level of aircraft operations and, correspondingly, the level of passenger volumes would have possibly been be reduced, marginally at first, but growing over time. It is also possible that without proper investment, the runway will fail altogether at some point. Since the Airport had only one runway, repairs could close the Airport for a period of 3 to 6 months, with extreme economic consequences.

C.2.4.1.2. Passenger Terminal Constraints

Without significant new investment, the physical structure of the passenger terminal building was going to continue to deteriorate, impacting the quality of service provided to passengers and increasing the economic and social costs to users.

MCC-15-RFQ-0095 Page 9 of 64

The deterioration of the physical structure could have resulted in additional operating expenses required for maintenance, electricity and energy, translating into higher airline rents that will be passed on to the traveler in the form of increased ticket prices. In a market with such low per capita income, incremental changes in ticket prices could have had material impact on the volume of passenger traffic, particularly in the intra-West African market.

As the number of passengers at the Airport increases, without expansion and improvement of the terminal building, there will be a clear deterioration in the level of service. Without investment, the same building that is estimated to currently operate beyond its capacity would have to accommodate over three times the present level of annual traffic within the next 20 years. In the future, this deteriorating level of service, combined with inadequate security, would have likely discouraged some airlines from maintaining or expanding services at the Airport and potentially inhibited new airlines from entering this market. The net result would have had the effect of de-stimulating demand for air travel and translate into fewer passengers being handled through the Airport.

Activities

The Airport activities to be covered under this evaluation include:

Airside Infrastructure: reinforcement overlay to and expansion of the runway, taxiways, and apron areas; replacement of deteriorating navigational equipment; and upgrades of Airport security systems.

Landside Infrastructure: improvements to the existing passenger terminal and construction of a new passenger terminal, as well as support facilities, airport roads, and parking lots. Certain utilities, including water supply, solid waste disposal facilities, wastewater treatment, and power generation were also to reconstructed and designed as joint systems to support both the proposed investments at the Airport and the adjacent Industrial Park.

Institutional Strengthening. In order to ensure the sustainability of the proposed investments, infrastructure improvements were accompanied by the establishment of appropriate institutional mechanisms that will ensure effective management, operation and maintenance of the Airport facilities over the long term. These measures will involve both the management of the Airport, as well as the wider regulatory framework governing the civil aviation sector in Mali.

Airside Infrastructure

Overview of Activity

MCC-15-RFQ-0095 Page 10 of 64

C.2.6.1.1. Although the pre-existing aircraft pavements, runway, and parking apron surfaces were functional, they were more than thirty years old and detailed studies have indicated that they would likely deteriorate without near-term improvement. In addition to being in poor condition, the runway was also one of the shortest in West Africa, which had further constrained the Airport’s ability to attract air services to Mali.

C.2.6.1.2. To address these issues, the Project:

Resurfaced, reinforced, and expanded the runway, apron, and aircraft pavement areas through: (i) an asphalt overlay of 12 cm; (ii) a 400 meter extension of the runway; (iii) an extension of the apron to provide a location for additional aircraft overnight staging and a back-up for smaller domestic and charter aircraft.

Replaced existing navigational aids to bring Airport facilities up to a “common level of service,” as the equipment has reached the end of its useful life. The extension of the runway will also require additions to the airfield lighting system.

Improved airfield security through gate area configuration changes, construction of modest guard shelters, a dedicated radio communication system to replace the current use of personal cell phones, installation of video surveillance equipment, an Airport security command post, and a computerized security pass system. A perimeter road security system will also be developed so that security forces can inspect the fence line and react to security concerns around the perimeter.

Landside Infrastructure

MCC-15-RFQ-0095 Page 11 of 64

C.2.7.1.1. The pre-existing Airport passenger terminal had not undergone any major improvements since 1974, with the exception of a separate international arrival facility (Terminal B) constructed in 2002. Due to limited expansion over the past 40 years, the ability of the terminal to accommodate passenger traffic has steadily deteriorated to the point that it operates at IATA LOS F (chronic congestion and frequent system breakdown). The ground support equipment facilities (GSE) were inefficient, outdated, and separate passenger activities from Airport support operations, with a resulting negative impact on the functionality and security of the Airport. As passenger and cargo traffic increases over the next 10-15 years, significant utility infrastructure improvements were needed to meet projected demand.

C.2.7.1.2. To address these issues, the Project:

Upgraded passenger terminal facilities by: (i) refurbishing the existing Terminal A’s passenger ticketing, lounge, and passport control areas; (ii) expanding the existing Terminal B’s immigration and baggage areas; and (iii) constructing a new passenger terminal building2, as maintaining terminal operations will be very difficult while simultaneously enhancing the existing facilities;

Enhanced support equipment such as GSE facilities, Airport maintenance and auxiliary equipment areas, and fire-fighting vehicles;

Provided for road and parking improvements to meet the future projected needs and to improve current circulation areas;

and

Constructed supporting utility infrastructure, to handle the projected service requirements of the Airport. In particular, wastewater, water, solid waste, power, telecommunications, and drainage systems will be improved and enhanced.

Institutional Strengthening

2 The construction of a new passenger terminal was not completed at the time the Compact was terminated.

MCC-15-RFQ-0095 Page 12 of 64

C.2.8.1.1. Under the present division of jurisdictions, a number of entities have responsibility for the civil aviation sector in Mali in general and the regulation, oversight, management, operation, and development of the Airport in particular. The Ministry of Equipment and Transport has overall responsibility, with oversight and regulation of the civil aviation sector and airports delegated to a new independent agency ANAC (Agence Nationale de l’Aéronautique Civile). The Bamako- Sénou Airport’s maintenance and operation responsibilities are split between the air navigation service provider, ASECNA, for airside facilities and the Airport operator, AdM, for landside facilities.

C.2.8.1.2. More specifically, ASECNA is responsible for the “technical” aspects of the Airport, including the runway, taxiways, apron, airfield lighting, navigational aids, control tower, telecommunications and fire fighting and rescue facilities. AdM, in turn, is responsible for the “commercial” aspects of the Airport, including the passenger terminal, landside roads and parking, cargo terminal, flight kitchen and freight forwarders’ facilities. There is no formal agreement between AdM and ASECNA regarding any specific physical line of demarcation concerning their respective responsibilities for maintenance at the Airport.

C.2.8.1.3. In response to ICAO Safety and Security Audits and FAA assessments, the GoM restructured and consolidated this institutional framework and established a new regulatory oversight agency, ANAC, in December 2005, which enjoys greater financial and administrative independence than its predecessor.

C.2.8.1.4. To address these issues, the Airport Project:

Reinforced the new civil aviation regulatory and oversight agency (ANAC) by providing technical assistance to establish a new organizational structure, administrative and financial procedures, staffing and training, and provision of equipment and facilities.

Rationalized and reinforced the Airport’s management and operations agency (AdM) by providing technical assistance to establish a model for the management of the Airport and the long-term future status and organizational structure of AdM.

At present, there is no single entity responsible for the management of the entire Airport, as both ASECNA and AdM have jurisdiction over airside and landside facilities, respectively. This has led to a diffuse decision-making process with respect to required capital investments and a resulting negative impact on facility maintenance. It has also contributed to the inadequate security environment at the Airport.

MCC-15-RFQ-0095 Page 13 of 64

These improvements to the management and regulatory framework of the Airport will help ensure the sustainability of the Airport Project over time.

Additional background information on the project can be found on the MCC and MCA Mali websites.

In cr ea se in H ou se ho ld

In co m e

Reinforcement/Expansion of Taxiway and Apron

The wage bill of firms servicing the Airport increases

Process/ Outputs

Short-term Outcomes

Medium-term

Outcomes

Long-term Outcomes

Replacement of Navigational Equipment

Upgrade to security systems

Poverty Reduction through Econom ic G row th

Increase in number of foreign visitors

New support facilities

New Terminal

Landside Infrastructure

Activity

Airport Management Training

Civil Aviation Regulatory Training

The wage bill in the tourism industry increases

Reinforcement/Expansion of Runway

Employment has been created

Airside Infrastructure

Activity

Increased number of flights

Improved services of the passenger terminal

Increases in the value and volume of goods shipped

Increased number of direct air connections

Institutional Strengthening

Increased Airport revenue generated by growth in passenger and aircraft traffic

Time Savings of passengers going through the airport

Increased demand for airline services

Maintenance of Navigational Equipment

Capacity to manage private Airport

Operator

Logic Assumptions

1. In order for there to be an increase in flights due to any Airport activity there must exist sufficient demand for airport services.

2. For time savings to occur for passengers utilizing the upgraded terminal systems airlines must accept the security systems as sufficient and not insist on additional security protocols.

MCC-15-RFQ-0095 Page 14 of 64

The general research questions to be answered by this performance evaluation include but are not limited to:

How sound was the problem diagnostic work in the project design process?

Were the correct activities implemented given the identified problems in airport capacity?

What were the barriers to implementation?

How sustainable are the benefits from the activities?

What are the primary lessons to be learned in future MCC decision making?

More specific research questions relating to:

Operations and maintenance on airside activities; and

Effectiveness of airport operations as a result of MCC interventions;

C.2.10.2.1. Will be developed in consultation with MCC and other stakeholders under Task One outlined below.

Roles and Responsibilities:

The MCC, Mission of Achievement of the Airport and Alatona Projects (or MAPPA who is acting as the Mali Accountable Entity), along with the independent evaluator, play critical roles in designing, implementing, and disseminating the independent evaluations. The MCC and MAPAA roles are as follows:

Millennium Challenge Corporation (MCC): MCC is responsible for oversight of the independent evaluator and quality control of evaluation activities, including the following specific responsibilities:

C.3.1.1.1. Set the evaluation priorities and evaluation questions. Using the program logic as the foundation, MCC will develop the evaluation priorities and evaluation questions based on existing evidence, the evidence gap, and the learning priorities for the sector and agency.

MCC-15-RFQ-0095 Page 15 of 64

C.3.1.1.2. Build buy-in and ownership of the evaluation. MCC will work with the independent evaluator and MAPAA to ensure the evaluation design follows the rules of program design and implementation. When possible, MCC will work with the independent evaluator and MAPAA to develop a program implementation roll-out plan that enables a rigorous evaluation design. When it is not possible to develop a coordinated roll-out plan, the evaluator will ensure the evaluation design is based on feasible methodologies given program design and implementation. MCC will work with MAPAA to ensure commitment to the roll-out plan by local stakeholders, and if is not possible to adhere to the roll-out plan, will inform the independent evaluator in order to adapt the evaluation design.

C.3.1.1.3. Determine the evaluation budgets. MCC will develop an evaluation budget, including independent evaluators, expected data collection requirements, and any other activities related to the evaluation.

C.3.1.1.4. Manage the independent evaluators. MCC will develop Terms of Reference for the independent evaluators and manage the contracting process, including Request for Proposals, Proposal Review, selection of the independent evaluator, development of a work plan, and supervision of the independent evaluator in line with the work plan and agreed deliverables.

C.3.1.1.5. Manage the data collection firms. As appropriate, MCC will develop Terms of Reference for the data collection firms, with technical support from MAPAA and the independent evaluator. In this scenario, the MCC is responsible for managing the contracting process, including Request for Proposals, Proposal Review, selection of the data collection firm, development of a work plan, and supervision of the data collection firm in line with the work plan and agreed deliverables.

C.3.1.1.6. Quality review of evaluation materials. Using the program logic as a foundation, MCC is responsible for reviewing evaluation materials, including the design report and survey materials, for feasibility, technical and factual accuracy, and adherence with international standards (when applicable).

MCC-15-RFQ-0095 Page 16 of 64

C.3.1.1.7. Quality review of evaluation reports. MCC is responsible for reviewing independent evaluators’ reports to ensure technical and factual accuracy. While MCC will provide feedback, the independent evaluator is ultimately the author of the evaluation reports and may or may not take into consideration MCC’s comments. MCC therefore develops an Official Statement for each evaluation report to confirm the technical and factual accuracy of the report, or clarify any differences on technical or factual issues between MCC and the independent evaluator.

C.3.1.1.8. Lead initial public dissemination efforts. MCC is responsible for leading initial public dissemination of results, in particular ensuring a feedback loop internally in order to inform future decision making on similar program investment, design, and implementation. This requires ensuring the evaluation results are disseminated on the MCC website, and results are disseminated to appropriate stakeholders in a timely fashion.

MAPAA: MAPAA is responsible for building local ownership and commitment to the evaluation and quality control of evaluation activities, including the following specific responsibilities:

C.3.1.2.1. Build buy-in and ownership of the evaluation. MAPAA will work with the independent evaluator and MCC to ensure the evaluation design follows the rules of program design and implementation. When possible, MAPAA will work with the independent evaluator and MCC to develop a program implementation roll-out plan that enables a rigorous evaluation design. MAPAA will work with MCC to ensure commitment to the roll-out plan by local stakeholders, and if it is not possible to adhere to the roll-out plan, will inform MCC in order to request the independent evaluator adapt the evaluation design.

C.3.1.2.2. Quality review of evaluation materials. Using the program logic as a foundation, MAPAA is responsible for reviewing evaluation materials, including the design report and survey materials, for feasibility, technical and factual accuracy, and adherence with international standards (when applicable).

C.3.1.2.3. Quality review of evaluation reports. MAPAA, or a designated representative, is responsible for reviewing independent evaluators’ reports to ensure technical and factual accuracy. While MAPAA will provide feedback, the independent evaluator is ultimately the author of the evaluation reports and may or may not take into consideration MAPAA’s comments. MAPAA therefore develops an Official Statement for each evaluation report to confirm the technical and factual accuracy of the report, or clarify any differences on technical or factual issues between MAPAA and the independent evaluator.

MCC-15-RFQ-0095 Page 17 of 64

C.3.1.2.4. Facilitate public dissemination efforts. MAPAA, or a designated representative, is responsible for facilitating public dissemination of results, in particular ensuring a feedback loop in-country in order to inform future decision making on similar program investment, design, and implementation.

Roles and Responsibilities

Independent Evaluator Roles and Responsibilities: The independent evaluator is responsible for the overall design, implementation, and dissemination of the evaluation, including the following specific responsibilities:

Develop a rigorous evaluation design given rules of implementation and feasibility of options. Using the program logic as the foundation, independent evaluators must assess the program design and implementation to develop the most rigorous evaluation design feasible, whether it is a performance or impact evaluation, and identify the most appropriate evaluation methodology feasible given the context. The independent evaluator is responsible for developing an evaluation design report, in consultation with MCC and MAPAA, which summarizes the evaluation design, sampling, analysis plan, and other critical design elements of the evaluation. (Tasks 1-2)

Support MCC and MAPAA to build buy-in and ownership of evaluation. The independent evaluator will meet with MCC and local stakeholders to develop the evaluation design based on program design and implementation. When possible, the independent evaluator will work with MCC and MAPAA to advise on a program implementation roll-out plan that enables a rigorous evaluation design. In addition, the independent evaluator will continuously present the evaluation objectives, materials, and interim and final results for MCC and local stakeholders to maintain commitment to the evaluation. (Task 2 and Task 7)

Develop evaluation materials that are held to international standards. Using the program logic as a foundation, the independent evaluators define how key outcomes will be measured and develop the survey instruments or other data collection tools in order to answer critical evaluation questions. Note that “survey” here refers primarily to quantitative surveys, but may also refer to qualitative instruments where appropriate (Task 3 and Task 8)

Ensure appropriate review of evaluation materials and research protocols.

Independent evaluators are responsible for ensuring all evaluation materials, including the evaluation design, survey instruments, sampling strategy, data collection and entry protocols are reviewed by an institutional review board (IRB). IRB review and approval is required before any data collection commences. The independent evaluators will document all approvals and informed consent procedures throughout the evaluation time period. (Task 4 and Task 9)

MCC-15-RFQ-0095 Page 18 of 64

Procure and manage the data collection firms. As appropriate, the evaluator will develop Terms of Reference for the data collection firms, with technical support from MCC and the MCC. In this scenario, the independent evaluator is responsible for managing the contracting process, including Request for Proposals, Proposal Review, selection of the data collection firm, development of a work plan, and supervision of the data collection firm in line with the work plan and agreed deliverables. (Task 4 and Task 9)

Supervise data collection. The independent evaluator is responsible for quality control over data collection activities, including interviewer training, pre-testing, and implementing data quality review checks during data collection.

(Task 4 and Task 9)

Lead data cleaning, analysis, interpretation of results. The independent evaluator is responsible for leading any necessary data cleaning and consultation with the data collection firm to produce final analysis files. The independent evaluator is responsible for leading all analysis in line with the agreed analysis plan developed with MCC, and consulting with MCC should the analysis plan need to be revised or adapted. (Task 5 and Task 10)

Produce evaluation reports. The independent evaluator is responsible for sharing initial evaluation reports with local stakeholders and MCC for review and feedback. However, given the independent evaluator is the author of the evaluation reports, local stakeholder and MCC feedback may or may not be taken into consideration for the final report. The independent evaluator is responsible for documenting all feedback and evaluator responses. (Task 5 and Task 10)

Lead public dissemination efforts. The independent evaluator is responsible for leading public dissemination efforts facilitated by MAPAA and MCC (such as local workshops and conferences), but also additional conferences or other opportunities to publicly disseminate the results of the evaluation. The evaluator will advise MCC on any public dissemination opportunities and collaborate as appropriate. (Task 11)

Tasks

Task 1 Assess Evaluation Plan (There must be approval on the deliverables associated with Task 1 before work on Task 2 can begin)

The Evaluator is responsible for reviewing the MCC Evaluation Plan, including the proposed Economic Rate of Return (ERR) analysis and beneficiary analysis, for the project/activity under evaluation. The Evaluator must assess:

C.5.1.1.1. Program Logic: Is the program logic evidence based with corresponding literature review and clearly defined risks and assumptions?

MCC-15-RFQ-0095 Page 19 of 64

C.5.1.1.2. Research Questions: By working with MCC and other stakeholders, work to finalize list of research questions and end uses for research questions. Are the research questions clearly defined, linked to the program logic, and feasible to answer?

C.5.1.1.3. Proposed Methodology (if applicable): If MCC has proposed an evaluation methodology, is the methodology appropriate and feasible?

MCC and the Evaluator should identify all MCC and MAPAA self-evaluation activities and determine how the independent evaluation will (i) validate findings of self-evaluation(s), (ii) use self-evaluation(s) to inform the independent evaluation, particularly around understanding what was designed, what was implemented, and understanding any necessary course corrections,

(iii) complement the self-evaluation(s) by expanding on self-evaluation activities.

The Evaluator will provide an Assessment Report with any recommendations to changes in the Program Logic, Research Questions, or Proposed Methodology

Develop a Work Plan with expected deadlines for deliverables. Any revisions to the Work Plan will be submitted in the Evaluator’s Progress Reports.

Task 2 Develop Evaluation Design Report

The Evaluator will conduct missions to the field as needed to discuss program design and implementation issues to inform the evaluation design, as well as engage local stakeholders (MAPAA and Implementing Entities, as well as local government) on the proposed evaluation. This includes hosting local stakeholder workshops to develop/refine research questions, assess appropriate and feasible quantitative and qualitative evaluation methodologies, develop evaluation timeline, and build local stakeholder commitment to the evaluation design.

The Evaluator will develop the Evaluation Design Report, including all technical analysis required for assessing and confirming internal validity, power calculations, and required sample sizes. The Evaluation Design Report outline will be developed in consultation with the MCC M&E lead. Please reference Annex 1 Evaluation Design Report Outline as a reference.

As per MCC’s Gender Policy and Gender Integration Guidelines, the Evaluator will ensure that evaluation design, data collection requirements, and analysis address gender and other social differences as appropriate. This involves including research questions that address social and gender dimensions of the project logic, assumptions, and project design;

disaggregating impacts by gender, age, income, and other social difference (e.g. ethnicity) as appropriate; including special modules or studies as needed.

MCC-15-RFQ-0095 Page 20 of 64

The Evaluator will document feedback from local stakeholders and the response to that feedback. In order to obtain feedback, the Evaluator may lead local workshops and/or virtual reviews of the Design Report by MAPAA and Implementing Entities.

C.5.2.4.1. In accordance with Section H, the Evaluator is responsible for allowing any key local stakeholders the opportunity to submit a public Statement of Difference or Support, as agreed with MCC and the MAPAA/designated representative.

The Evaluator will submit the Evaluation Design Report to MCC and may be asked to present to the Evaluation Management Committee. All feedback and response to feedback will be documented by the Evaluator. Any revisions to the report deemed necessary by the Evaluator will be incorporated into the final draft.

As per Section H, MCC may: (i) select to obtain external peer review; and (ii) make the Report and peer reviews public on the MCC website.

The Evaluator will continue to maintain the Evaluation Design Report as the evaluation develops. The Evaluation Design Report will remain a living document that is updated by the Evaluator as necessary and resubmitted to MCC as necessary.

Task 3 Develop Evaluation Materials

The Evaluator will provide technical support on development of data collection firm TORs.

The Evaluator will lead development of any necessary survey instruments for data extraction from existing sources or data collection for new sources.

The Evaluator will develop survey materials in English, unless otherwise agreed with MCC. When necessary, the Evaluator will utilize existing instruments to improve coordination across MCC independent evaluations for meta-analysis. The Evaluator is responsible for any necessary pilot testing of survey instruments and review of back-translations conducted by data collection firms prior to data collection start.

The Evaluator is responsible for ensuring household surveys last no longer than 120 minutes on average without written consent from the Evaluation Management Committee. The Evaluator is responsible for reporting on the average survey time resulting from the pilot test and from field work.

The Evaluator will document feedback from local stakeholders and response to the feedback. In order to obtain feedback, the Evaluator may lead local workshops and/or virtual reviews of the Evaluation Materials by MAPAA and Implementing Entities.

MCC-15-RFQ-0095 Page 21 of 64

The Evaluator will submit the Evaluation Materials to MCC and may be asked to present to the Evaluation Management Committee. All feedback and response to feedback will be documented by the Evaluator. Any revisions to the report deemed necessary by the Evaluator will be incorporated into the final draft.

C.5.3.6.1. The Evaluator is responsible for the quality of the English and local language survey instruments.

Task 4 Prepare and Supervise baseline data collection

The Evaluator is responsible for developing and overseeing all data collection and entry protocols and implementation. This requires ensuring appropriate Institutional Review Board (IRB) review and approval of research protocols, survey instruments, and informed consent procedures (please reference the MCC Informed Consent Statement Template Annex 2).

C.5.4.1.1. In accordance with Section H, the Contractor should assess the local IRB requirements and determine whether or not additional, international-level IRB approval is required in order to ensure adherence to international standards.

The Evaluator is responsible for overseeing the training of data collection staff, field work, and data entry. This must be managed in coordination with MCC and MAPAA.

The Evaluator is responsible for conducting and documenting quality control checks of data as it is collected and entered in order to reduce data collection and entry error.

The Evaluator is responsible for establishing a system for frequent communication with the data collection firm throughout data collection and entry, through email, phone, and site visits.

Task 5 Develop Baseline Report and Data Files

The Evaluator will review raw files from the survey firm and provide any critical feedback on production of the survey firm’s Data Collection Report.

Using the raw dataset, the Evaluator will produce a cleaned analysis dataset that follows MCC’s guidelines for public use data, see Annex 3 - MCC Data Documentation and Anonymization Requirements.

C.5.5.2.1. This will include STATA do files and final merged analysis files for documentation purposes.

MCC-15-RFQ-0095 Page 22 of 64

In addition, the Evaluator will produce a full set of metadata for each survey that follows MCC’s guidelines for public use data, see Annex 3 - MCC Data Documentation and Anonymization Requirements.

The Evaluator is responsible for developing the Baseline Report. The Baseline Report outline will be developed in consultation with the MCC M&E lead.

Please reference Annex 5 Baseline Report Outline as a reference.

The Evaluator will ensure baseline analysis including gender and social analysis is part of the evaluation design as described in the evaluation design report (task 2), including examining differences in baseline characteristics by gender, age, income and other social variables as appropriate.

The Evaluator will document feedback from local stakeholders and response to feedback. In order to obtain feedback, the Evaluator may lead local workshops and/or virtual reviews of Baseline Report by MAPAA and Implementing Entities.

The Evaluator will submit the Baseline Report to MCC and present to the Evaluation Management Committee. All feedback and response to feedback will be documented by the Evaluator. Any revisions to the report deemed necessary by the Evaluator will be incorporated into the final draft.

As per Section H, MCC may: (i) select to obtain external peer review; and (ii) make the Report and peer reviews public on the MCC website

Task 6 Disseminate Baseline Results

Once the final report is reviewed by the Evaluation Management Committee, the Evaluator is responsible for:

C.5.6.1.1. Presentation at MCC headquarters

C.5.6.1.2. Presentation at MAPAA/accountable entity headquarters

The Evaluator may also be responsible for reviewing any materials developed by MCC public relations for dissemination on the MCC website for quality assurance.

The Evaluator may be asked to participate in other MCC-financed dissemination and training events, such as MCC M&E College and MCC Impact Evaluation Workshops

Task 7 Monitor program implementation and conduct risk assessment

MCC-15-RFQ-0095 Page 23 of 64

The Evaluator is responsible for continuous monitoring of program implementation to ensure implementation occurs according to design and any deviations from design are properly documented. This requires collaboration with MCC, MAPAA sector leads and implementing entities to ensure continuous support for and commitment to the evaluation design.

For impact evaluations, this requires a minimum of quarterly status updates of implementation according to the treatment (to ensure the intervention is implemented in the agreed treatment areas) and control (to ensure the intervention is not implemented in the agreed control areas).

The Evaluator is responsible for flagging any serious concerns or risks regarding the internal validity of the evaluation throughout this period.

Task 8 Revise Interim/Final Evaluation Materials

The Evaluator is responsible for any necessary updates and revisions to the Interim/Final Evaluation Materials, including questionnaires and training materials, based on lessons learned.

The Evaluator is responsible for ensuring household surveys last no longer than 120 minutes without written consent from the Evaluation Management Committee. The Evaluator is responsible for reporting on the average survey time resulting from the pilot test and from field work.

The Evaluator will document feedback from local stakeholders and any response to feedback. In order to obtain feedback, the Evaluator may lead local workshops and/or virtual reviews of the Evaluation Materials by MAPAA and Implementing Entities.

The Evaluator will submit the Evaluation Materials to MCC and present to the Evaluation Management Committee. All feedback and response to feedback will be documented by the Evaluator. Any revisions to the report deemed necessary by the Evaluator will be incorporated into the final draft.

Task 9 Prepare and Supervise interim/final data collection

The Evaluator is responsible for developing and overseeing all data collection and entry protocols and implementation. This requires ensuring appropriate Institutional Review Board (IRB) review and approval of research protocols, survey instruments, and informed consent procedures (please reference the MCC Informed Consent Statement Template Annex 2).

C.5.9.1.1. In accordance with Section H, the Contractor should assess the local IRB requirements and determine whether or not additional, international-level IRB approval is required in order to ensure adherence to international standards.

MCC-15-RFQ-0095 Page 24 of 64

The Evaluator is responsible for overseeing the training of data collection staff, field work, and data entry. This must be managed in coordination with MCC and MAPAA.

The Evaluator is responsible for conducting and documenting quality control checks of data as it is collected and entered in order to reduce data collection and entry error.

The Evaluator is responsible for establishing a system for frequent communication with the data collection firm throughout data collection and entry, through email, phone, and site visits.

Task 10 Develop Interim/Final Report and Data Files

The Evaluator will review raw files from the survey firm and provide any critical feedback on production of the survey firm’s Data Collection Report.

Using the raw dataset, the Evaluator will produce a cleaned analysis dataset that follows MCC’s guidelines for public use data, see Annex 3 - MCC Data Documentation and Anonymization Requirements.

C.5.10.2.1. This will include STATA do files and final merged analysis files for documentation purposes.

In addition, the Evaluator will produce a full set of metadata for each survey that follows MCC’s guidelines for public use data, see Annex 3 - MCC Data Documentation and Anonymization Requirements.

The Evaluator is responsible for data analysis and developing the Interim/Final Report. The Interim/Final Report outline will be developed in consultation with the MCC M&E lead. Please reference Annex 6 Interim/Final Report Outline as a reference.

The Evaluator will ensure that…