SOW for posting to FBO.doc

DRAFT STATEMENT OF WORK

IMPACT EVALUATION

NEW ECONOMIC OPPORTUNITIES (NEO) PROJECT

I. Summary

The NEO project is a four-year, $20.6-million activity with start and end dates of April 2011-April 2015. NEO’s purpose is to improve rural incomes, reduce poverty levels, improve food security, and address critical, small-scale household and agricultural water constraints in targeted communities. Additionally, NEO will enable targeted internally displaced persons (IDP) to sustainably maintain their households and assist communities distressed by natural or other disasters.

NEO supports USAID/Georgia’s assistance objective of improved economic competitiveness and welfare and its intermediate results: improved private sector competitiveness, improved economic security of targeted vulnerable populations and sectors, and improved economic infrastructure in strategic sectors.

The proposed evaluation will run until 2015 and will assess NEO’s impact on rural incomes, household poverty levels; and community-level planning and economic infrastructure development processes in target communities.

II. Background

The NEO contract was awarded to Chemonics International in April 2011. Chemonics is implementing NEO in collaboration with their partners, International Relief and Development (IRD) and the National Opinion Research Center (NORC) at the University of Chicago.

The implementing partner is using a causal pathway methodology as a tool for their strategic approach, which envisions identifying impact statement and then working backward chronologically to define activities that produce the desired impact, NEO being a “development facilitator” in this process.

The impact statement as defined by the implementing partner reads as follows: “Sustainable poverty reduction, improved living standards for vulnerable populations and increased government participation in addressing local community needs.”

NEO’s activities are organized according to four components, including community-level economic development planning (LED); rural economic development; assistance to strengthen highly vulnerable households and individuals; and promoting the sustainability of IDP houses being rehabilitated with support from the USG. In addition, NEO has built-in small disaster response mechanism.

LED planning and economic strengthening activities represent the core of the program and are designed to complement each other. Within selected municipalities, NEO will apply a three-pronged approach in support of sustainable local economic development:

1. LED planning processes will serve as an entry point for NEO and help identify economic sectors, prioritize infrastructure and other investments, facilitate public-private dialogue, and leverage additional investments. NEO will accomplish this by mobilizing a network of stakeholders – local government and central government representatives, working groups and informal local leaders, donors, implementers, private sector – and establish coordination mechanisms and targeted LED events to foster dialogue and joint action.

2. NEO’s economic strengthening activities are designed to address the needs of vulnerable segments of the population. Working in the value chains identified in the economic development plans, NEO will provide targeted assistance to facilitate vertical and horizontal linkages, increase access to finance and provide market driven training. These activities will be driven by market demands, economic principles and best practices.

3. In addition, NEO will provide micro-grants and technical assistance to highly vulnerable households to strengthen their food security. These activities may fall outside those sectors or value chains identified as “high potential” but may be implemented because of their potential impact on impoverished households. Beneficiaries will be selected based on a set of criteria developed to determine their vulnerability.

IDP housing activities were designed as stand-alone activities, as they might not align geographically with selected municipalities. In cases where there is geographic overlap, beneficiaries of IDP housing activities will be fully integrated into other NEO activities as per established criteria.

NEO’s disaster response is not an on-going activity but rather a mechanism that can be triggered by USAID should the need arise.

NEO will provide assistance in 10 municipalities (approximately 84 communities within these municipalities) over the life of the project selected in coordination with USAID, the U.S. Embassy, and the Government of Georgia (GoG). Work will begin in three municipalities during the first year and will expand to other municipalities during FY 2012. The municipalities were chosen based upon the following weighted criteria

· High population of disadvantaged (60 percent)

· Concentration of IDPs per municipality (30 percent)

· Proximity to conflict zones and/or impacted by the 2008 conflict (10 percent) NEO plans to conduct three surveys such as baseline, mid-project and final-project household outcome and citizen satisfaction surveys to collect baseline and later measure the project’s results . The evaluation team will be able to use this data; however will need to collect new/additional data based on the sample to be selected by the evaluator.

III. Purpose of the Evaluation and Its Intended Use The purpose of this impact evaluation is to assess:

a. whether NEO’s support for community/municipal-level planning process and economic infrastructure development activities increased the voice of communities in municipal decision-making and resulted in economic impact on affected communities;

b. whether NEO’s rural economic development initiatives, including capacity building interventions and value chain assistance, improved rural incomes;

c. whether assistance to vulnerable households alleviated poverty levels by increasing productivity or creating jobs.

The evaluation team shall complete the study in three phases, the first tentatively planned for March 2012; second - for December 2013; and third - for May 2015. NEO commenced in April 2011, municipalities were selected, and selection of communities is under way. The proposed date for the first phase of the evaluation (mid-FY 2012) will fit well into the overall schedule of the project with regards to collecting baseline information, identifying sources of secondary data to be used as baseline for select municipalities, and select “treatment” and “control” communities. NEO staff, the external evaluation team, and the USAID mission will work together to coordinate data collection, monitoring and analysis as it overlaps for project monitoring and impact evaluation purposes.

Since part of the NEO funding is coming from the $1 billion pledge, evaluation results will be used for accountability purposes both to the USG and Government of Georgia. In addition, this will be a learning experience since lessons learned throughout the evaluation process will contribute to the ongoing life of project implementation of NEO. The final results of the evaluation will help the mission to define future activities and approaches for community economic development planning, poverty alleviation and increase in rural incomes. These results will be shared widely within the E&E region.

IV. Evaluation Questions and Methodology Key evaluation questions:

1. How effective and sustainable was the community and municipality economic development planning methodology and approach developed and used by the project? To what extent did the project result in: (a) incorporating community-level economic development priorities into higher-level municipal economic development plans and (b) leveraging GOG and/or other donor funding to finance the implementation of these plans (e.g. construction of economic infrastructure)?

2. What was the economic impact or change of income status of community members in a benefiting community as a result of the small infrastructure projects and in-kind procurements (e.g. farming equipment)?

3. What was the overall impact of NEO’s rural economic development component (value chain assistance) on increasing incomes and creating jobs in targeted communities? To what degree did the component increase productivity and/or profitability of targeted farms/businesses?

4. What was the impact of providing grants vs. other types of assistance as a means of addressing project goals?

5. What was NEO’s impact on increasing access to financial services for underserved agricultural and non-agricultural rural producers/processors/service-providers?

6. What was the resulting impact of micro-grants, in-kind support, cash-for-work and capacity-building interventions provided to highly vulnerable households toward sustainably alleviating poverty (e.g., an increase in productivity / the creation of sustainable micro-entrepreneurs, or providing one-time spike in consumption)?

7. Did the project affect men and women in the communities differently? (the evaluator should incorporate into research and provide sex-disaggregated data, where possible, such as women-headed households, etc.)

Offerors shall partner with a local organization/s and shall suggest the best methods that minimize bias and provide strong evidence. While experimental designs generate the strongest evidence for impact, given the project design, municipality selection process, and timeline, quasi-experimental methods are likely to be more realistic. One possible methodology could be a difference-in-difference methodology in which a number of “treatment” communities from those targeted by the project are compared to non-affected “control” communities. The appropriate number of communities to include in the sample and the method for selecting those communities shall be proposed by the evaluation team.

Various data collection and analysis methods, both quantitative and qualitative, shall be used, including surveys (to supplement project-collected data where needed to answer impact questions) and secondary data sources (including official country statistical information), interviews, and focus groups (to get qualitative information on community member perceptions around the issues of the study). When possible, NEO-collected baseline data shall be used to avoid duplication in data collection.

The offeror shall coordinate with the NEO implementing partner with regards to data collection. However, since the evaluation team will be selecting communities for the study, the offeror shall collect their own data for their sample, which they shall do through their local partner organization to be cost-effective. The prime contractor shall take the lead in survey design and data analysis.

Responses to the question #1 may be obtained partly by a review of annual financing trends for project-assisted vs. other communities.

Answers to the question #7 shall be obtained using interviews and focus groups.

Data collection and analysis approaches shall be further elaborated by the offeror and proposed to USAID based on the methodology for this evaluation. Also, the confidence level, sample size, and sampling methodology shall be proposed.

V. Evaluation Team Requirements While the offeror can propose the composition of the Team, the following are key personnel skills required for the completion of the evaluation (this does not cover locally hired staff to collect data):

The Team Leader (international) will have a minimum of fifteen-year experience conducting evaluations and assessments including impact evaluations in economic growth and poverty reduction areas. The expert should have at least master’s level education in the field of economics. Experience in Georgia and/or in the Europe and Eurasia region is highly desirable. The team leader will be responsible for the day to day management of the team, data collection and synthesis, presentations and draft and interim/final reports.

Evaluation Expert (international) will have a minimum of ten-year experience conducting evaluations and assessments including impact evaluations using various experimental, quasi-experimental and other methodologies. The expert should have extensive experience in statistics and statistical research and methodologies, should be able to provide advice and participate in developing a research plan, including sampling, sample size, selection methods as well as statistical analysis of collected data.

Two locally hired experts/consultants provided by a local partner organization with a minimum of five-year experience in economic and poverty reduction fields, as well as the experience of participating as a team member in conducting a USAID project-related assessment and/or evaluation. English language knowledge is a requirement. Collectively, the team should have education and experience with economics and statistical methods.

The incumbents should have considerable experience in designing, monitoring and evaluating development assistance programs. They must have excellent written and oral presentation skills.

The evaluation contractor needs to make sure that the team composition is preserved throughout the evaluation contract life (unless the change is necessary due to low quality work); in case of unavailability of one or more consultants during the entire period, the contractor needs to ensure substitution of expert/s with similar level qualifications.

The prime contractor and its local partner organization will decide how to provide translation, transportation and logistical support to the evaluation team.

All Team members will be required to provide to USAID a signed statement attesting to a lack of conflict of interest.

VI. Deliverables

The evaluation contractor shall conduct three (three/four-week) visits to Georgia. The team shall include local partner organization experts. Below is an illustrative plan for each visit, however, the offeror shall propose their own plan for each visit in their proposal.

During the first field work (March – April 2012):

The important tasks of this visit are to develop the detailed evaluation plan for all three visits, to make sample of municipalities for research and to set baseline. The report will be submitted on the activities of the component I. The design will have detailed description of research methodology including its strengths and limitations. The plan will include evaluation matrix – each evaluation question with respective methodology to collect information, information source, etc.

1. Create and submit research design and work plan

2. Select “treatment” and “control” municipalities.

3. Plan and collect baseline information including baseline survey (coordinate with the project plans); set baseline for the

4. Initiate qualitative field work

5. Gather useful secondary data

6. Agree with NEO leadership on the need for collecting monitoring data collection for the use by the evaluation.

Local partner:

Local partner organization will collect baseline data, work with international partner on evaluation design of evaluation, secondary data gathering as well as working with the NEO project implementer on sharing monitoring and baseline data that the implementer will have. In between the visits the local partner will be following up as needed on data collection and serving as a resource on the ground for international partner.

1. Carry out baseline survey

2. Follow program evolution

3. Review monitoring data and follow the data collection

4. Work with NEO staff on refining monitoring data collection, if needed.

5. Submit report After the first visit the contractor will submit a detailed evaluation plan (a research design, as well as the results of the work conducted). The contractor will also be required to submit separate reports after each of the first two visits within 10 days after the completion of each visit.

Second visit (November-December 2013):

The purpose of the second visit will be to review the research plan, make adjustment if needed, collect data and analyze it to draw the preliminary conclusion of project progress and results.

1. Review existing data

2. Adapt research plan to changes in program focus and/or locations

3. Conduct follow-up data collection and qualitative field work

4. Combine results to draw preliminary conclusions

5. Submit report Third and final visit (tentatively March 2015):

The purpose of the third visit will be to conduct the last part of the study to make conclusions on the project impact with regards to the evaluation questions listed under chapter IV of the current document. Close collaboration with USAID and NEO implementer is expected during all visits.

1. Conduct final part of the impact evaluation

2. Collect final data

3. Analyze data

4. Identify impact of Neo project with regards to the key evaluation questions outlines in the SOW.

5. Submit draft evaluation report

6. Finalize the report based on stakeholder review comments.

The evaluation contractor will submit the final report within 20 days after the completion of the third and final visit.

Evaluation Reports should meet the following criteria:

· Evaluation reports shall address all evaluation questions included in the statement of work.

· The evaluation report should include the statement of work as an annex. Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report.

· Evaluation findings will assess outcomes and impact on males and females.

· Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).

· Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence.

· Sources of information shall be properly identified and listed in an annex.

· Recommendations shall be supported by a specific set of findings.

· Recommendations shall be action-oriented, practical and specific, with defined responsibility for the action.

Major sections of each report should be:

· An executive summary: 3-5 pages in length summarizing the purpose, background of the project being evaluated, main evaluation questions, methods, findings, conclusions, and recommendations and lessons learned (if applicable);

· Table of contents;

· An evaluation report clearly delineating finding, conclusions, recommendations and lessons learned (see criteria below);

· Appendices.

The appendices to the report shall include:

· The evaluation statement of work;

· Any “statements of differences” regarding significant unresolved difference of opinion by funders, implementers, and/or members of the evaluation team;

· All tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides;

· Sources of information, properly identified and listed;

· Disclosure of conflicts of interest forms for of all evaluation team members;

· The evaluation design.

Deliverables

Component I

· Detailed research design and evaluation work plan

· Evaluation matrix

· Submit report for component I

· Implement baseline survey/report

· Review monitoring data and follow the data collection

· Select “treatment” and “control” municipalities

· Plan and collect baseline information including baseline survey

· Gather useful secondary data

Component II

· Adapt/refine research plan if needed

· Submit report for Component II - preliminary conclusions of evaluation

· Conduct follow-up data collection and qualitative field work

· Review existing and collected data

Component III

· Final evaluation report

VII. Other Requirements

The Evaluation team shall demonstrate familiarity with USAID’s Human Subject Protection Policy and USAID’s Evaluation Policy (http://www.usaid.gov/evaluation). The Evaluation team shall provide adequate training for its survey staff on survey methodology, USAID’s survey regulations, other relevant regulations, and data collection plan.

The Contractor has the responsibility to safeguard the rights and welfare of human subjects involved in the survey research supported by USAID. USAID has adopted the Common Federal Policy for the Protection of Human Subjects, Part 225 of Title 22 of the Code of Federal Regulations (http://www.usaid.govb/policy/ads/200/200mbe.pdf). Recipient organizations must familiarize themselves with the USAID policy and provide “assurance” that they will follow and abide by the procedures of the Policy.

All records from the evaluation (e.g., interview transcripts or summaries) must be provided to the Contracting Officer’s Technical Representative (COTR). All quantitative data collected by the evaluation team must be provided in an electronic file in easily readable format agreed upon with the COTR. The data should be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID will retain ownership of the survey and all datasets developed.

� Selected municipalities include: Oni, Tsageri, Kazbegi, Lentekhi, Kareli, Dusheti, Khashuri, Gori, Tsalenjikha and Zugdidi.

� Neo Year I Workplan – will be shared with the evaluation team once selected

� Some Outcome level indicators to be tracked by NEO (final indicators and targets will be shared with the evaluation team):

% increase in average value of targeted household production, % of targeted vulnerable households and individuals raised to the official subsistence level

% change in average household incomes in targeted communities and sectors

% increase in number of adult individuals that perceive that the local government understands and is responsive to their needs