RFQ_-_Needs_Annexes-_Burkina_Agriculture_Evaluation_TOR_v4_-_SOW2-1-FINAL-3.docx

PAGE 1 OF

1. REQUISITION NO.

2. CONTRACT NO.

3. AWARD/EFFECTIVE DATE

4. ORDER NO.

5. SOLICITATION NUMBER

6. SOLICITATION ISSUE DATE

a. NAME

b. TELEPHONE NO. (No Collect Calls)

8. OFFER DUE DATE/LOCAL

TIME

9. ISSUED BY

CODE

10. THIS ACQUISITION IS

UNRESTRICTED OR

SET ASIDE:

% FOR:

SMALL BUSINESS

HUBZONE SMALL

BUSINESS

SERVICE-DISABLED

VETERAN-OWNED

SMALL BUSINESS

WOMEN-OWNED SMALL BUSINESS

(WOSB) ELIGIBLE UNDER THE WOMEN-OWNED

SMALL BUSINESS PROGRAM

EDWOSB

8(A)

NAICS:

SIZE STANDARD:

11. DELIVERY FOR FOB DESTINA-

TION UNLESS BLOCK IS

MARKED

SEE SCHEDULE

12. DISCOUNT TERMS

13a. THIS CONTRACT IS A

RATED ORDER UNDER

DPAS (15 CFR 700)

13b. RATING

14. METHOD OF SOLICITATION

RFQ

IFB

RFP

15. DELIVER TO

CODE

16. ADMINISTERED BY

CODE

17a. CONTRACTOR/OFFEROR

CODE

FACILITY CODE

18a. PAYMENT WILL BE MADE BY

CODE

TELEPHONE NO.

DUNS:

DUNS+4:

PHONE:

FAX:

17b. CHECK IF REMITTANCE IS DIFFERENT AND PUT SUCH ADDRESS IN OFFER 18b. SUBMIT INVOICES TO ADDRESS SHOWN IN BLOCK 18a UNLESS BLOCK BELOW IS CHECKED

SEE ADDENDUM

19.

20.

21.

22.

23.

24.

ITEM NO.

SCHEDULE OF SUPPLIES/SERVICES

QUANTITY

UNIT

UNIT PRICE

AMOUNT

(Use Reverse and/or Attach Additional Sheets as Necessary)

25. ACCOUNTING AND APPROPRIATION DATA

26. TOTAL AWARD AMOUNT (For Govt. Use Only) 27a. SOLICITATION INCORPORATES BY REFERENCE FAR 52.212-1, 52.212-4. FAR 52.212-3 AND 52.212-5 ARE ATTACHED. ADDENDA

ARE

ARE NOT ATTACHED.

27b. CONTRACT/PURCHASE ORDER INCORPORATES BY REFERENCE FAR 52.212-4. FAR 52.212-5 IS ATTACHED. ADDENDA

ARE

ARE NOT ATTACHED

28. CONTRACTOR IS REQUIRED TO SIGN THIS DOCUMENT AND RETURN _______________

29. AWARD OF CONTRACT: REF. ___________________________________ OFFER

COPIES TO ISSUING OFFICE. CONTRACTOR AGREES TO FURNISH AND

DATED ________________________________. YOUR OFFER ON SOLICITATION

DELIVER ALL ITEMS SET FORTH OR OTHERWISE IDENTIFIED ABOVE AND ON ANY

(BLOCK 5), INCLUDING ANY ADDITIONS OR CHANGES WHICH ARE

ADDITIONAL SHEETS SUBJECT TO THE TERMS AND CONDITIONS SPECIFIED

SET FORTH HEREIN IS ACCEPTED AS TO ITEMS:

30a. SIGNATURE OF OFFEROR/CONTRACTOR 31a. UNITED STATES OF AMERICA (SIGNATURE OF CONTRACTING OFFICER) 30b. NAME AND TITLE OF SIGNER (TYPE OR PRINT) 30c. DATE SIGNED 31b. NAME OF CONTRACTING OFFICER (TYPE OR PRINT) 31c. DATE SIGNED

AUTHORIZED FOR LOCAL REPRODUCTION

(REV. 2/2012)

PREVIOUS EDITION IS NOT USABLE

Prescribed by GSA - FAR (48 CFR) 53.212

7. FOR SOLICITATION

INFORMATION CALL:

STANDARD FORM 1449

OFFEROR TO COMPLETE BLOCKS 12, 17, 23, 24, & 30

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL ITEMS

20160150

MCC-16-RFQ-0096

perryr@mcc.gov 04-28-2016

Millennium Challenge Corporation Contracts and Grants Management Division 1099 Fourteenth Street NW Suite 700 Washington DC 20005

N/A X

Millennium Challenge Corporation

CGM

1099 Fourteenth Street NW, Suite 700 Washington DC 20005

Millennium Challenge Corporation Contracts and Grants Management Division 1099 Fourteenth Street NW Suite 700 Washington DC 20005

Interior Business Center Interior Business Center M/S D-2773 7301 West Mansfield Avenue Lakewood CO 80235-2230

See CONTINUATION Page Evaluation in support of the Agriculture Development Project Compact

Evaluation to design and implement evaluations of Water Management and Irrigation and the Diversified Agriculture activities in Burkina Faso

See CONTINUATION Page

X

SECTION D PACKAGING AND MARKING 26

SECTION E INSPECTION AND ACCEPTANCE………………………………………………………………………………………………………………………………….…..……..……26

M.3. Technical Evaluation Citeria…………………………………………..……………………………

Independent design and implementation evaluation services in support of the Water Management and Irrigation and the Diversified Agriculture activies in Burkina Faso.

Firm-Fixed Price Base Period (Phase I) POP TBD (Estimated 6-8 months)

Independent design and implementation evaluation services in support of the Water Management and Irrigation and the Diversified Agriculture activities in Burkina Faso.

Cost Reimbursement Base Period (Phase I) POP TBD (Estimated 6-8 months)

Sub-Total:

Independent design and implementation evaluation services in support of the Water Management and Irrigation and the Diversified Agriculture activies in Burkina Faso.

Time and Material Option Period I (Phase II) POP TBD (Estimated 12-18 months)

Independent design and implementation evaluation services in support of the Water Management and Irrigation and the Diversified Agriculture activies in Burkina Faso.

Time and Material Option Period I (Phase II) POP TBD (Estimated 12-18 months)

Sub-Total:

Independent design and implementation evaluation services in support of the Water Management and Irrigation and the Diversified Agriculture activies in Burkina Faso.

Time and Material Option Period II (Phase III)

Independent design and implementation evaluation services in support of the Water Management and Irrigation and the Diversified Agriculture activies in Burkina Faso.

Time and Material Option Period II (Phase III)

Sub-Total:

GRAND TOTAL

B.2 Accounting and Appropriation Data

[RESERVED]

B.3 Labor Rates Labor Rates at the inception of a Base Period or an Option Period will carry through as “Fixed-Price” without adjustment for the life of that Base Period or Option Period.

B.4 Pricing/Task Order Type The Base Period will be Firm Fixed Price.

Option Periods I and II will be Time and Materials. (MCC reserves the right to negotiate a firm fixed price with the successful offeror for Phases II and III prior to exercising each option period.

B.5 Other Direct Costs (ODC) Offerors shall include all propsed Other Direct Costs (ODCs) for each project phase identified in B.1 and detailed in Sections C.4. through C.4.12.5.

B.6 Limitation of Funds – Incrementally Funded Contracts In accordance with clause MCC52.232-72, “Limitation of Funds – Incrementally Funded Contracts,” funds in the amount of [fill in amount at time of award] have been allotted to this contract. It is contemplated that funds now obligated under this contract will cover the work to be performed until [fill in date at time of award].

The Contractor shall not perform work on the contract which exceeds the total amount actually allotted by the Government to the contract. The Government is not obligated to reimburse the Contractor for costs incurred in excess of the total amount allotted by the Government to this contract. The contractor assumes the risk for any increased costs beyond what the Government obligates.

B.7 Contract Type

(End of Provision)

(a) The Government contemplates award of a Time and Materials/Labor Hour contract for option periods I and II of performance resulting from this solicitation.

(b) The offeror must specify fixed houly rates in its offer that include wages, overhead, general and administrative expenses, and profit. The offeror must specify whether the fixe hourly rate for eachlabor category applies to labor peformed by_

(1) The offeror;

(2) Subcontractors; and/or

(3) Divisions, subsidiaries, or affiliates or the offeror under a common control.

(End of Provision)

(End of Section B)

DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK

C.1 Scope MCC seeks a new Independent Evaluator in support of the Agriculture Development Project (ADP) of the Burkina Faso Compact, which closed on July 31st, 2014.

C.2 Background and Objectives MCC seeks an Independent Evaluator to design and implement evaluations of Water Management and Irrigation and the Diversified Agriculture activities of the Agriculture Development Project in the Burkina Faso Compact. It is anticipated that quantitative methods will be supplemented by applicable qualitative methods in order to best address the range of questions being posed under this evaluation.

C.2.1. Project Summary

C.2.1.1. The Agriculture Development Project of the Burkina Faso Compact was designed to ease some of the constraints on economic growth in the country, namely limited access of producers to skilled labor, markets and inputs, water resources, and credit. The objective of MCC’s investment in the sector was to address these constraints in order to expand the productive use of land, increasing the volume and value of agricultural production in the Project zones. The ADP consisted of three main activities:

C.2.1.2. The Water Management and Irrigation Activity ($103.9M) This activity ensured adequate water availability, water delivery, flood control, and dam safety to support and protect investments in the Sourou Valley and Comoé Basin. It included construction of a 2,240 hectare irrigated perimeter in Di, rehabilitation of the Léry Dam, establishment of organizations of water users in irrigated perimeters, capacity building for the Autorité de Mise en valeur de la Vallée du Sourou (AMVS) and the Direction Générale des Ressources en Eau (DGRE) within the Ministry of Agriculture, and the adoption into law of Integrated Water Resource Management Master Plans for the Mouhoun and Comoé basins along with the creation of a Basin Agency and Local Water Committees for each to implement the basin’s Master Plan.

C.2.1.3. The Diversified Agriculture Activity ($29.7M) This Activity supported on-farm agro-sylvo-pastoral production and related activities throughout the value chain, focusing on, but not limited to producers who are beneficiaries of the Water Management and Irrigation component of the Project. Implemented in the Sourou Valley and in the Comoé Basin, it included technical assistance and training for rainfed and irrigated agriculture, animal husbandry, administrative and technical training for value-chain actors, expansion of a market information system, and rehabilitation of four rural markets.

C.2.1.4. The Access to Rural Finance Activity ($6.8M) This activity was designed to increase access to credit for small and medium enterprises working in agriculture value chain activities in the four western regions of Burkina Faso (Boucle du Mouhoun, Cascades, Hauts Bassins, and Sud-Ouest). It included creation of a Rural Finance Facility (RFF) to offer financing for medium-term loans to farmers and small and medium agricultural enterprises in the target regions, training of the Participating Financial Institutions (PFIs) which extended the loans via RFF funding, and Business Development Services to potential end borrowers to reinforce their capacity to manage their finances, thereby reducing perceived risk of the PFIs and any other financial institution considering expansion into the sector. Note that after delayed startup and slow loan uptake this Activity was closed in mid-2013.

C.2.2. The following program logics explain in more detail the primary components of the Agriculture Development Project.

C.2.3. Evaluation Summary

C.2.3.1. The evaluation services sought under this SOW would cover the following ADP activities:

1. Water Management and Irrigation:

a. The new irrigated perimeter at Di

b. Integrated Water Resource Management (IWRM)

c. Capacity Building and Technical Assistance for Operations & Maintenance in the Sourou

2. Diversified Agriculture

a. Farmer Training

b. Value Chain Development

c. Animal Health Services

d. Market Information System

e. Rehabilitation of Rural Markets

C.2.3.2. MCC procures two different types of evaluations: Impact Evaluations (which include an estimation of the counterfactual, such as in RCTs or Difference-in-Difference designs) and Performance Evaluations (which include both quantitative and qualitative designs where there is not an estimation of the counterfactual).

C.2.3.3. Prior to this call for new evaluation services, three distinct, but related, evaluations had previously been planned for the above ADP activities (see detailed evaluation plans in Annex I- Previous Evaluation Design Report).

1. A randomized control trial covering 710 of the total 2240ha on the new perimeter at Di (“The Di Lottery”).

2. A performance evaluation covering 1099 of the total 2240ha on the new perimeter at Di provided to Project Affected Persons (PAPs) who were provided land as compensation through the project (“The Di PAP Evaluation”).

3. A difference-in-difference evaluation with a sample of 1000 treatment/1000 comparison households was planned to cover the farmer training, value chain development, and animal health services activities (“The Farmer Training Evaluation”).

C.2.3.4. It is anticipated that the Evaluator will plan additional methods for the following evaluations (which had not been a part of the prior design):

4. A performance evaluation of the IWRM and Capacity Building and Technical Assistance for Operations and Maintenance sub-activities (“The Water Management Evaluation”)

5. A performance evaluation of the Market Information Systems and Rehabilitation of Rural Markets sub-activities (“The Rural Markets Evaluation”)

C.2.3.5. Related to the above listed evaluations, several data collection efforts have already been undertaken (surveys with an “*” indicate that a data quality review report is available for review in Annex II- Data Quality Reviews):

1. For the Di Lottery:

a. A baseline survey of all lottery participants in Dec 2013 (land was distributed via lottery in Feb 2014)*

2. For the Di PAP Evaluation:

a. Data was collected in order to estimate PAP compensation in 2011

b. There was additional data collected on the PAPs in 2012

c. A sample of PAPs was then surveyed again in 2013, just as some PAPs began receiving their new land plots on the perimeter

3. For the Farmer Training Evaluation:

a. A baseline survey was conducted in 2012 on the 1000 treatment/1000 comparison sample (as well as some additional data related to other project components- see annex)*

b. Bovine weight data was collected in 2012 and 2013

c. Crop yield data was collected from the 1000 treatment/1000 comparison households in 2013*

4. For the Water Management Evaluation:

a. A few modules of the 2012 baseline survey in 3.a. above included information related to the management and operations of Water User Associations, which may be useful for the Water Management Evaluation*

5. For the Rural Markets Evaluation:

a. A limited amount of data was collected during the course of project finalization

C.2.3.6. As described in section C.3, during Phase I the Evaluator will be expected to assess the prior evaluation design, project implementation, as well as the baseline data in order to provide a revised and updated design for the final stages of the evaluation. This review is anticipated to require significant due diligence on the part of the Evaluator to assess the prior evaluation design, assess the previously collected data, determine the evaluation’s feasibility, and recommend any necessary changes or additions. This first phase of work will result in a revised evaluation design and work plan. Phase II is anticipated to include the implementation of interim data collection and completion of the Interim Evaluation Report. Phase II will also include the completion of baseline reports and the anonymization of the previously collected datasets, as set out in section C.3. Phase III is anticipated to include the final data collection and completion of the Final Evaluation Report. However, the specific activities (such as surveys or qualitative data collection) included within these phases (and their timing) may be altered, depending on the final Design Report provided by the Evaluator. The offeror is expected to provide an estimate for the complete work of all phases in their proposal. However, the contract will be incrementally funded for each of the three periods, and the Contractor shall wait for a written directive from MCC before beginning on future Phases or Parts of Phases under the contract.

C.2.3.7. It is expected that the results from the interim data collection will inform whether or not it is pertinent for MCC and the Evaluator to continue forward with the final data collection. Thus, a thorough assessment of short and medium term outcomes will be critical for the interim data collection. Based on the results found, an assessment will be made about how and whether to move forward with the final data collection.

C.2.3.8. There are currently several known constraints or challenges related to each of the evaluations:

1. For the Di Lottery Evaluation

a. Not all training services to new farmers on the perimeter were completed prior to the end of the Compact. These services were to be continued by the government post-Compact. An assessment will be needed to acertain the extent to which farmers have actually received this training and whether or not variations in the type or duration of training is likely to pose a threat to the evaluation.

2. For the Di PAP Evaluation:

a. Collecting reliable income data for the PAP households was known to be problematic. The first sets of surveys done were done with the purpose of compensation, and thus it is believed that some estimates may have been overstated in order to maximize compensation. The income data collected in the 2013 survey only collected income information from some sources, and thus provides only a partial picture of PAP incomes. Thus, fully assessing income from a quantitative perspective may be problematic, an issue that will need to be assessed by the Evaluator.

3. For the Farmer Training Evaluation:

a. The initial 1000 treatment and 1000 comparison households were chosen before it was known who would actually benefit directly from the various activities. An analysis in 2013 was done, and it was found that only about 620 of the 1000 treatment households were actually treated. The Evaluator will need to assess the impact of this on the evaluation and plan a path forward.

4. For the Water Management Evaluation and Rural Markets Evaluation:

a. Limited baseline data was collected. Thus, it is expected that evaluation of these components may be primarily qualitative.

C.2.3.9. MCC also anticipates that several changes (in addition to those proposed by the Evaluator) to the prior evaluation design may be needed, including:

1. Inclusion of an assessment of the viability of each evaluation methodology, given the potential issues noted above.

2. Removal of the Economic Rate of Return and Beneficiary Analysis components from the prior evaluation methodology

3. Inclusion of an assessment of the program logic and presentation of ways the program logic could have been improved.

4. Elaboration of methodologies to address the Water Management and Rural Markets Evaluations.

5. Addition of qualitative methods to 1- complement the existing quantitative methods, 2- address research questions that cannot be otherwise addressed using quantitative data, and 3- provide additional depth of understanding regarding project results and effects.

6. The Evaluator shall examine the feasibility (and cost-effectiveness regarding potential learning) of examining the following indirect impacts. A final decision on whether or not to include these questions in the Evaluation will be jointly decided by MCC and the Evaluator.

a. Adoption or adaptation of the technological innovations promoted by the project by others outside the target group;

b. Adoption or expansion of institutional and structural changes (such as implementation of the CATG and the WUAs) beyond the boundaries of the project intervention areas.

7. In terms of data collection rounds, the former evaluation methodology had anticipated an interim survey round starting in 2015 and a final survey round starting in 2016. Both rounds were anticipated to last (one) 1 full year. Given the timing of this solicitation, as the basis of their proposals, offerors should assume that interim data collection will begin in 2017 and endline data collection will begin in 2018. While endline data collection should be assumed to take a full year (which would include multiple rounds to cover the multiple cropping seasons), interim data collection should be assumed to include a single round instead of a full year. Section F provides an estimated timeline for these activities.

C.2.3.10. The anticipated research questions (which may vary from the Design Report in Annex I) are listed below. The Evaluator will be expected to work with MCC, the post-MCA government entity known as the APD, and local stakeholders to assess and finalize this list as a part of the elaboration of the Evaluation Design Report. To the extent possible, these questions shall also include an assessment of why or why not changes were observed, whether the observed changes are likely to be sustainable, as well as an analysis as to how (and why) effects may vary by gender.

C.2.3.10.1 Overall Questions Regarding the Project:

1. How sound and realistic was the program logic underpinning the Agriculture Project? In what ways could it have been improved?

2. The original program logic for the Agriculture Project anticipated a cohesive and complimentary project made up of numerous components. During implementation, how well did this complimentarity hold true?

3. To what extent did implementation follow the intended project design? And how did this influence outcomes and the cohesiveness of the different project components?

4. Did the assumptions made in this program logic hold true? How did this influence outcomes?

C.2.3.10.2. Questions Regarding the Di Lottery, Di PAP, and Farmer Training Evaluations:

C.2.3.10.2.1. Research Questions that Apply to All Three Evaluations:

1. Agriculture production:

a) Has the amount of land under production and/or the level of production intensity (# of crop rotations/parcel of land) increased, decreased or remained the same for project participating farmers with respect to non-participating farmers? Why? Can the change be attributed to project activities?

b) How has the selection of crops changed? How do the shifts relate to crop prices? Have farmers move towards more high-value crops?

c) Has the average value of inputs used in production per hectare increased, decreased or remained the same for participating farmers compared to non-particpating farmers? Why? If there is difference, can it be attributed to project interventions?

d) Has the average total factor productivity of participating farmers increased, decreased or remained the same as compared to non-participating farmers? If there is a difference, why did this occur? Can the difference be attributed to project interventions?

e) What is the current average yield/ha for each of the focus crops: rice, corn, onions, tomatoes, soybeans and cowpeas? Have participating farmer average yields/ha increased, decreased or remained the same for each of the focus crops as compared to non-participating farmers? Why? Can the change be attributed to project activities?

f) Does access to irrigation affect yields, total production? sales, and household income?

2. Household income

a) Has the average percentage of commercialized production increased, decreased or remained the same for participating farmers compared to non-particpating farmers? If there is a difference, what is the difference and why is it occurring? Can the difference be attributed to project interventions?

b) Have participating household’s agricultural sales increased as a result of the project? Has this resulted in increased agricultural net income?

i. If yes, does this lead to higher average total household incomes?

c) Did the project affect sales, total farm income and/or household income?

d) To what extent has the project had an effect on employment in rural areas?

e) Has there been a change in the components of household income (and/or in their distribution)? If so, what change occurred and why?

C.2.3.10.3. For the Di Lottery Evaluation:

1. Land Tenure Security:

a) Have all beneficiaries received the land tenure instruments (titles or leases) they were informed they would receive?

b) How has the perception of land security changed?

c) Has the change in land security affected land investments?

d) Has the change in land security affected land sales and/or rental markets?

e) Have beneficiaries successfully or unsuccessfully attempted to use their new land tenure documents as collateral for loans to fund agriculture investments (and/or other investments)?

2. Cost-Effectiveness:

a) What were the costs relative to the benefits on the irrigated perimeter? How do they compare with other agricultural interventions?

3. Lottery Criteria:

a) Standard parcel sizes were identified for rice and polyculture lands in the lottery, which were based on the understanding that these sizes would maximize benefits for households. Did this assumption hold true?

4. Soil erosion

a) To what extent has soil erosion decreased or increased as a result of project supported IWRM and O&M interventions?

5. Soil fertility

a) To what extent has soil organic matter compostion, salinity and compaction improved or deterioted as a result of project supported IWRM and O&M interventions?

6. Landscaping

a) To what extent have the landscapes introduced to to maintain MCC infrastructure investments (such as grass plantings along canals) been maintained or enhanced as a result of project supported IWRM interventions?

C.2.3.10.4. For the Di PAP Evaluation:

1. Land Tenure Security:

a) How has the perception of land security changed?

b) Have all beneficiaries received the land tenure instruments (titles or leases) they were informed they would receive? Why or why not?

c) Has the change in land security affected land investments?

d) Has the change in land security affected land sales and/or rental markets?

e) Have beneficiaries successfully or unsuccessfully attempted to use their new land tenure documents as collateral for loans to fund agriculture investments (and/or other investments)?

2. Restoration of Livelihoods:

a) How has PAP financial well-being changed? Are PAPs at least as well of after the project as they were before?

b) Have any PAPs been harmed (socially, economically or politically) by the intervention? How?

c) Have all PAPs received the compensation instruments (titles and/or leases and/or financial compensation) they were informed they would receive?

d) What are the PAPs’ perceptions of the compensation provided and the process by which it was determined and then provided?

e) What are the PAPs’ perceptions of their land tenure status on the new perimeter (which included formalized land rights) C.2.3.10.5. For the Farmer Training Evaluation:

a) Rainfed and Irrigated Agriculture Technical Assistance:

b) Have farmers adopted/adapted improved production practices proposed by the project (e.g., compost production and use, pesticide and chemical fertilizer use, improved seed use, improved planting and harvesting techniques, crop rotation, etc.) into their farm operations?

c) Are participating farmers correctly using the improved production practices as presented during the training, or are they adapting them to local context? If adaptation is occurring, what improved farming practices have been modified and why?

d) Which improved production practices were adopted/adapted the most? Which were adopted/adapted the least? Why? If some were not adopted/adapted at all, why not?

e) What proportion of the average participating farmer’s farm operation uses components of the training?

f) Are (or did) participating farmers use the incentive kits that they received along with training?

g) Are (or have) the incentive kits having (or had) an impact on adoption of improved farming techniques? Crop yields? Have farmers continued to invest in improved seeds/fertilizers?

h) Are participating farmers diversifying crop production more or less than farmers that did not benefit from the project?

2. Animal Husbandry Practices:

a) Have farmers adopted improved animal husbandry practices proposed by the project into their farm operations?

b) Are participating farmers correctly using the improved animal husbandry practices as presented during the training, or are they adapting them to local context? If adaptation is occurring, what improved animal husbandry practices have been modified and why?

c) Which improved animal husbandry practices were adopted/adapted the most? Which were adopted/adapted the least? Why? If some were not adopted/adapted at all, why not?

d) What proportion of the average participating farmer’s farm operation uses components of the training?

e) Are farmers vaccinating their animals (chickens and cows)?

f) Have livestock vaccination rates increased or decreased as a result of training?

g) How has project supported vaccination work affected livestock mortality (chickens and cows)?

h) Are farmers using artificial insemination for cattle? If so, what is the rate of successful births and what are the weight gains? If not, why not?

3. Agriculture Value Chain Development:

a) To what extent are Producer Associations established under the project functioning? How many continue to operate? What factors have contributed to their level of operation?

b) Of the Producer Associations that received value-addition training, how many are using components of the training? How many are successfully adding value to their products?

c) Are participapting Producer Associations correctly using value-addition techniques as presented during the training, or are they adapting them to local context? If adaptation is occurring, what componens of the training have been modified and why?

d) Which value-addition training components were adopted/adapted the most? Which were adopted/adapted the least? Why? If some were not adopted/adapted at all, why not?

4. Animal Health Services:

a) To what extent are the veterinary schools and labs that were rehabilitated still functioning? Have the facilities been properly maintained?

b) What has become of the veterinarians trained? Are they still providing services to the farmers in project intervention zones?

c) Do farmers have better access to veterinarian services as a result of project interventions?

d) How has demand for veterinary sevices changed as a result of project activities?

e) Has livestock health and productivity increased as a result of project interventions?

C.2.3.10.6. Questions Regarding the Water Management Evaluation

1. Integrated Water Resource Management:

a) To what extent are SDAGEs being implemented as planned? What are the primary factors influencing their operation?

b) How well are the CLE and Basin Committee institutions functioning? What are the primary factors influencing their operation?

c) Are user/polluter fees (i.e. Contribution Financière en matière d’Eau (CFE)) fully defined, and are they being collected? To what extent? If so, are the funds being directed to the CLEs and Basin Committees or into the national treasury?

d) Has IWRM work had an effect on water conflicts?

e) To what extent have water resources (ground and surface) benefitted (e.g., improved conservsation, increased use efficiency, increased quantity, better quality, etc.) or been harmed (e.g., increased flooding, drought conditions, reduction in quality, etc.) as a result of project supported IWRM and O&M interventions?

2. O&M Technical Assistance and Capacity Building for Irrigation in the Sourou

a) To what extent are irrigation systems that benefitted from project activities being effectively operated and maintained? Is their level of operation sustainable?

b) Has the Government of Burkina Faso continued to implement the AMVS reform strategic plan that was developed during the Compact? If so, to what extent? If not, why not?

c) To what extent is the AMVS successfully assuming its O&M responsibilities? Why or why not?

d) To what extent is the Centre d’Appui Technique et Gestion (CATG) operational? Why or why not?

i. If the CATG is functional, what percentage of WUAs are availing themselves of the services, and what services are they accessing ? What benefits do the WUAs perceive to using the CATG?

ii. If the percentage of WUAs using the CATG is low, why are so few using it?

e) To what extent are water user associations (WUAs) on the old Niassan perimeter and the new Di perimeter demonstrating the capacity (financial, technical, and organizational) to fully and sustainably leverage the irrigation investments at their disposal? What are the factors influencing their level of capacity/implementation?

C.2.3.10.7. Questions Regarding the Rural Markets Evaluation:

1. Market Information System (MIS):

a) To what extent are the two Information Centers that were created using project resources still operational? Why? Is their operation sustainable?

b) To what extent is the MIS still functional? How many commodities are being tracked using the system (and how has the number of commodities changed over time)?

c) How many users (farmers, producer groups, etc) are accessing the MIS market data?

d) Are farmers using the system to make more informed production and marketing decisions? Why or why not?

e) To what extent have transaction and marketing costs for farmers changed as a result of the MIS? Why have they changed or not changed?

f) Has using the MIS had an impact on net farm income?

2. Rural Market Rehabilitation:

a) How well are the market management committees functioning? What are the primary factors related to their functioning?

b) Has health, welfare, safety and sanitation improved within the 9 markets?

c) How has construction/rehabilitation of the 4 markets impacted their operation, size, or level of economic activity?

d) To what extent are the rehabilitated facilities being occupied and used? Who is using them? Are women benefiting? Why or why aren’t they being used?

e) Has market rehabilitation work contributed to increased net farm income? Why or why not, and to what extent?

f) To what extent has market rehabilitation work helped to increase the competitiveness of agricultural products through improved and rationalised processing and marketing of agricultural products?

g) To what extent has market rehabilitation work helped to increase the added value and competitiveness of agricultural products by improving their quality?

h) To what has market rehabilitation work improved the situation of the basic agricultural production sector?

C.3. Roles and Responsibilities:

C.3.1 The MCC, APD, along with the independent evaluator, play critical roles in designing, implementing, and disseminating the independent evaluations. The MCC and APD roles are as follows:

C.3.1.1. Millennium Challenge Corporation (MCC): MCC is responsible for oversight of the independent evaluator and quality control of evaluation activities, including the following specific responsibilities:

C.3.1.2. Set the evaluation priorities and evaluation questions. Using the program logic as the foundation, MCC will develop the evaluation priorities and evaluation questions based on existing evidence, the evidence gap, and the learning priorities for the sector and agency.

C.3.1.3. Build buy-in and ownership of the evaluation.

C.3.1.4. Determine the evaluation budgets. MCC will develop an evaluation budget, including independent evaluators, expected data collection requirements, and any other activities related to the evaluation.

C.3.1.5. Manage the independent evaluators. MCC will develop Terms of Reference for the independent evaluators and manage the contracting process, including Request for Proposals, Proposal Review, selection of the independent evaluator, development of a work plan, and supervision of the independent evaluator in line with the work plan and agreed deliverables.

C.3.1.6. Quality review of evaluation materials. Using the program logic as a foundation, MCC is responsible for reviewing evaluation materials, including the design report and survey materials, for feasibility, technical and factual accuracy, and adherence with international standards.

C.3.1.7. Quality review of evaluation reports. MCC is responsible for reviewing independent evaluators’ reports to ensure technical and factual accuracy. While MCC will provide feedback, the independent evaluator is ultimately the author of the evaluation reports and may or may not take into consideration MCC’s comments. MCC therefore develops an Official Statement for each evaluation report to confirm the technical and factual accuracy of the report, or clarify any differences on technical or factual issues between MCC and the independent evaluator.

C.3.2.1. Through this review process, MCC’s role is to enable the independent evaluator’s independence in order to produce a high quality, unbiased evaluation of the program. MCC should ensure that various stakeholders understand that while MCC, APD, and other stakeholders may provide guidance and leadership on evaluation questions and feasibility of methodology, once an approved evaluation design is in place, feedback on analysis and interpretation of results is limited to statements on the continued technical and factual accuracy of the analysis.

C.3.2.2. Facilitate public dissemination efforts. MCC is responsible for facilitating public dissemination of results, in particular ensuring a feedback loop in-country in order to inform future decision making on similar program investment, design, and implementation.

C.3.2.3. Agence de Partenariat pour le Développment (APD) and/or other designated representative of the Government of Burkina Faso: APD, the Post-Compact Entity, is responsible for building local ownership and commitment to the evaluation, oversight of the data collection firm, and quality control of evaluation activities, including the following specific responsibilities:

C.3.2.3.1. Build buy-in and ownership of the evaluation.

C.3.2.3.2. Quality review of evaluation materials. Using the program logic as a foundation, APD is responsible for reviewing evaluation materials, including the design report and survey materials, for feasibility, technical and factual accuracy, and adherence with international standards.

C.3.2.3.3. Quality review of evaluation reports. APD, or a designated representative, is responsible for reviewing independent evaluators’ reports to ensure technical and factual accuracy. While APD will provide feedback, the independent evaluator is ultimately the author of the evaluation reports and may or may not take into consideration APD’s comments. APD therefore develops an Official Statement for each evaluation report to confirm the technical and factual accuracy of the report, or clarify any differences on technical or factual issues between APD and the independent evaluator.

C.3.2.3.4. Facilitate public dissemination efforts. APD, or a designated representative, is responsible for facilitating public dissemination of results, in particular ensuring a feedback loop in-country in order to inform future decision making on similar program investment, design, and implementation.

C.4. Tasks C.4.1. Independent Evaluator Roles and Responsibilities: The independent evaluator is responsible for the overall design, implementation, and dissemination of the evaluation, including the following specific responsibilities.

C.4.1.1. Develop a rigorous evaluation design given rules of implementation and feasibility of options. Using the program logic and the previous evaluation design as the foundation, independent evaluators must assess the program design and implementation to develop the most rigorous evaluation design feasible, whether it is a performance or impact evaluation, and identify the most appropriate evaluation methodology feasible given the context. The independent evaluator is responsible for developing an evaluation design report, in consultation with MCC and APD, which summarizes the evaluation design, sampling, analysis plan, and other critical design elements of the evaluation. (Tasks 1-2) C.4.1.2. Support MCC and APD to build buy-in and ownership of evaluation. The independent evaluator will meet with MCC and local stakeholders to develop the evaluation design based on program design and implementation. In addition, the independent evaluator will maintain external commitment to the evaluation by continuously making evaluation objectives, materials, and interim and final results available to MCC and local stakeholders. (Task 2 and Task 7) C.4.1.3. Develop evaluation materials that are held to international standards. Using the program logic as a foundation, the independent evaluator will define how key outcomes will be measured and develop the survey instruments or other data collection tools in order to answer critical evaluation questions. Note that “survey” here refers primarily to quantitative surveys, but may also refer to qualitative instruments where appropriate (Task 3 and Task 8) C.4.1.4. Ensure appropriate review of evaluation materials and research protocols. Independent evaluators are responsible for ensuring all evaluation materials, including the evaluation design, survey instruments, sampling strategy, data collection and entry protocols are reviewed by an institutional review board (IRB). IRB review and approval is required before any data collection commences. The independent evaluators will document all approvals and informed consent procedures throughout the evaluation time period. (Task 4 and Task 9) C.4.1.5. Implement all data collection efforts, including the recruitment and management of the data collection firms. The evaluator will develop Terms of Reference for the data collection firms, with technical support from MCC and the APD. The independent evaluator is responsible for managing the contracting process, including Request for Proposals, Proposal Review, selection of the data collection firm, development of a work plan, and supervision of the data collection firm based on the work plan and agreed deliverables. (Task 4 and Task 9) C.4.1.6. Implement all data collection, ensuring proper quality control at all levels. The independent evaluator is responsible for quality control over data collection activities, including interviewer training, pre-testing, and implementing data quality review checks during data collection. (Task 4 and Task 9) C.4.1.7. Lead data cleaning, analysis, interpretation of results. The independent evaluator is responsible for leading any necessary data cleaning and consultation with the data collection firm to produce final analysis files. The independent evaluator is responsible for leading all analysis in line with the agreed analysis plan developed with MCC, and consulting with MCC should the analysis plan need to be revised or adapted. (Task 5 and Task 10) C.4.1.8. Produce evaluation reports. The independent evaluator is responsible for sharing initial evaluation reports with local stakeholders and MCC for review and feedback. However, given the independent evaluator is the author of the evaluation reports, local stakeholder and MCC feedback may or may not be taken into consideration for the final report. The independent evaluator is responsible for documenting all feedback and evaluator responses. (Task 5 and Task 10) C.4.1.9. Through this review process, the independent evaluator’s role is to assert independence in order to produce a high quality, unbiased evaluation of the program. The independent evaluator should ensure that various stakeholders understand that while MCC, APD, and other stakeholders may provide guidance and leadership on evaluation questions and feasibility of methodology, once an approved evaluation design is in place, feedback on analysis and interpretation of results is limited to statements on the continued technical and factual accuracy of the analysis.

C.4.1.10. Lead public dissemination efforts. The independent evaluator is responsible for leading public dissemination efforts facilitated by APD and MCC (such as local workshops and conferences), but also additional conferences or other opportunities to publicly disseminate the results of the evaluation. The evaluator will advise MCC on any public dissemination opportunities and collaborate as appropriate. As per Section H, the independent evaluator is responsible for ensuring that all public release documentation is reviewed to minimize any risk to respondent confidentiality. This may require submission of internal use only and public release reports. (Task 11) Base Period (PHASE I) C.4.2. Task 1: Assess Evaluation Plan C.4.2.1. The Evaluator is responsible for conducting an Evaluability Assessment on all existing documentation related to the program and proposed evaluation plan provided by MCC and APD using the MCC Evaluability Assessment Tool .

C.4.2.2. The Evaluator’s review and completion of the Evaluability Assessment Tool shall also include a review and examination of the program logic, including the assumptions and linkages in the logic.

C.4.2.3. The Evaluator will assess the previous evaluation design, including a review of the existing baseline data, the status of treatment and comparison groups, and the feasibility of continuing with the previous evaluation design. The Evaluator shall also examine the prior evaluation design and identify additional methods/analyses to strengthen or enhance the design.

C.4.2.4. In addition to assessment and modification of the previous evaluation design, the Evaluator will be expected to design evaluation methodologies to address the Water Management Evaluation questions and the Rural Markets Evaluation questions.

C.4.2.5. The Evaluator, with assistance from MCC and APD shall identify all MCC and APD self-evaluation activities and determine how the independent evaluation will (i) validate findings of self-evaluation(s), (ii) use self-evaluation(s) to inform the independent evaluation, particularly around understanding what was designed, what was implemented, and understanding any necessary course corrections, (iii) complement the self-evaluation(s) by expanding on self-evaluation activities.

C.4.2.6. The Evaluator will provide an Evaluability Assessment Report on the Problem Diagnostic, Program Logic, Risks & Assumptions, Proposed Research Questions, Methodology, and other Metrics C.4.2.7. Develop a Work Plan with expected deadlines for deliverables. Any revisions to the Work Plan will be submitted in the Evaluator’s Progress Reports.

C.4.3. Task 2: Develop Evaluation Design Report C.4.3.1. The Evaluator will conduct missions to the field as needed to discuss program design and implementation issues to inform evaluation design, as well as engage local stakeholders (APD, Implementing Entities, as well as local government) on the proposed evaluation. This includes hosting local stakeholder workshops to develop/refine research questions, assess appropriate and feasible quantitative and qualitative evaluation methodologies, develop evaluation timeline, and build local stakeholder commitment to the evaluation design.

C.4.3.2. The Evaluator will develop the Evaluation Design Report, including all technical analysis required for assessing and confirming internal validity, power calculations, and required sample sizes. The Evaluation Design Report outline will be developed in consultation with the MCC M&E lead. Please reference Annex III - Evaluation Design Report Outline as a reference.

C.4.3.3. In addition to revising the evaluation methodologies, the Evaluator will also propose a revised schedule for interim and final data collection. These revised dates will be documented in the Work Plan (or a revision to the Work Plan).

C.4.3.4. As per MCC’s Gender Policy and Gender Integration Guidelines (available on the MCC…