Attachment_7-_Task_Order_2__Ex_post_evaluation.pdf

USAID/Serbia - The role of community development and citizen engagement activities in strengthening civic engagement and government responsiveness in Serbia

Ex-Post Evaluation Statement of Work

I. Introduction

In preparation for the development of new activities promoting civic participation and engagement, USAID/Serbia is soliciting an ex-post evaluation of its key citizen engagement programs implemented since 2001. The purpose of the evaluation is to determine how past USAID interventions have contributed to sustainably increasing citizen engagement in public policy and government oversight.

This evaluation will focus on three USAID programs: the Community Development through Democratic Action (CRDA) Program; the Serbian Local Government Reform Program (SLGRP) and the Civil Society Advocacy Initiative (CSAI). The evaluation will be conducted over a 90 day period, resulting in a final evaluation report.

II. Background

USAID experience globally points to the critical role of citizen engagement can play in building more transparent, accountable, effective democracies. Research has shown that investment in strengthening civic engagement can be transformative in redefining democratic culture - both for the governed and the governing; it promotes a sense of civic responsibility and agency for citizens, and can contribute to building a culture of accountability, public service, and responsiveness in public officials.

As Serbia embarked on democratic reforms in the early 2000s, an influx of international donor assistance sought to build on the invigorated democratic activism in the country and channel that momentum into strengthening mechanisms for participatory, transparent, and accountable democracy.

In 2001, USAID/Serbia launched the Community Revitalization through Democratic Action (CRDA) Program, Serbia’s Local Government reform (SLGRP) Program and later, in 2006, the Civil Society Advocacy initiative (CSAI) Program to improve citizen participation in decision making process, introduce democratic good governance practices, increase citizen engagement with local and national governments and foster the growth of civil society in Serbia.

CRDA (2001 to 2006) was a five-year, $200 million program covering all of Serbia except for metropolitan Belgrade. It was a civil society program that used community development activities to build trust between different ethnic and religious groups, to demonstrate the value of citizen participation, to support grass roots democratic action and to bring immediate improvement in people's living conditions. During implementation (2001 to 2006), CRDA implemented over 5,000 projects throughout Serbia. Local communities contributed with almost 45% in matching funds or in-kind contributions. The CRDA portfolio included activities targeting four main pillars: civic participation, income generation, environmental protection, and infrastructure improvements with two special earmarks for reproductive health (RH) and assistance to refugee/IDP populations.

SLGRP (2001 to 2006), was a $30 million program active in 90 municipalities throughout Serbia. The combined population of these participating municipalities accounted for more than 80 percent of the total population of Serbia. Training sessions attended by more than 20,000 participants were held to improve the management capacities of local government staff. More than half of these participants were women, reflecting the important role that they played in local government administration. The training focused on financial management, public services management, information technology applications, and citizen participation. As a result of these efforts, transparent and accountable budgeting and financing systems were established in 70 municipalities and 65 participating towns held public budget hearings for the first time in Serbia. Public-private partnerships that created business improvement districts (BID zones) were introduced for the first time in Serbia. Citizen assistance centers and/or one stop permitting centers were set up in 30 municipalities.

CSAI - in 2006, USAID launched the Civil Society Advocacy Initiative (CSAI) as a $12 million, five-year program to help civil society organizations (CSOs) better represent citizen needs, and for civil society to become a more influential and trusted partner of businesses and the government. After three additional ceiling increases during 2008-10 and an extension of two years, the program ended in May 2013 as a seven-year, $27.5 million initiative. CSAI’s theory of change was that fundamental cultural and political change results from both citizens who are empowered to act and organizations that are capable of leading them. CSAI promoted civil society advocacy through grants, training and tailored technical assistance to develop individual, project and/or institutional capacity to exert influence and represent constituency interests. CSAI also undertook efforts to strengthen the civil society sector by removing obstacles in the operating environment and helping NGO coalitions to improve their effectiveness.

Despite these investments, recent public opinion polling illustrates low levels of public faith in governance institutions as well as a declining number of citizens who prefer democracy to any other form of government (the overall share of citizens who prefer a democratic form of governance is now 12% less than was the case in 2005), whereas there is a visible and growing number of those polled who believe non-democratic governments can be better than democratic ones.1 Simultaneously, without clear entry points or channels for changing the status quo or holding the government accountable for poor performance, citizens have abdicated responsibility for making change; other public opinion research has found that in many cases the state is perceived as the problem, but the state is also seen as the only entity with the power to generate solutions.2

USAID is seeking through this assessment to understand how key USAID counterparts were able to engage citizens and increase government responsiveness in the transition period of the early 2000s, and how and why some interventions have sustained over time and others have not.

I. Past Approaches to Citizen Engagement

In 2001, when USAID launched activities in Serbia, Serbia’s people and government faced numerous social and economic problems. In addition to the new challenges emerging in the process of the post-conflict transition, assistance was needed to strengthen still fragile democratic processes, revitalize the ruined economy, stimulate the nascent private sector, and increase the transparency and accountability of governing institutions. Citizen engagement was weak and USAID searched for options to develop civil

1 http://www.cesid.rs/uploads/lt/usaid/cesid-predstavio-rezultate-najnovijeg-istra-ivanja-javnog-mnjenja-o-politi-koj-situaciji-u-srbiji/CeSID_Konferencija-19.04.2016.pdf 2 http://www.cesk.org.rs/index.php/en/projects/14-projects/15-social-and-cultural-capital-in-serbia society development programs that would engage citizens in addressing these important social and economic problems. These programs included a variety of approaches, including, but not limited to:

1. Mobilizing citizens to implement community infrastructure projects to improve quality of life, bolster social capital and trust between religious and ethnic groups, and demonstrate the benefits of peaceful democratic governance.

2. Joint community development activities with citizens and local government to increase trust between citizens and the state

3. Establishment of Community Development Committees (CDC) to strengthen citizen participation with local government, through initiatives such as open budget hearings and citizen advisory councils to monitor service delivery.

4. Establishment of business improvement districts to revitalize the private sector and encourage economic growth

5. Creation of citizen advisory centers and one-stop permitting centers to make government services more accessible

6. Formation of advocacy coalitions that brought together like-minded advocacy organizations to promote policy reform

7. Awareness raising activities to promote a positive perception of civil society

8. Grants and training to civil society organizations to increase their capacity to represent citizen interests

9. Public awareness campaigns and legislative advocacy to promote individual and corporate philanthropy

10. Advocacy for State-CSO cooperation, including transparency in state funding for CSOs

The contractor will explore these, and other approaches taken by USAID partners, to determine their effectiveness in engaging citizens as well as the durability of project results over time.

III. Draft Ex-post Evaluation Questions

This ex-post evaluation will look at the different activities and approaches employed by these three programs in differing environments, with a view to identifying those that proved to be particularly effective at the time, which continued to exist post-USAID assistance, and why these interventions were effective.

The evaluation should consider the following:

1. Which approaches have had positive sustainable effects on civic participation, and which did not? Why?

2. Which approaches have had positive, sustainable effects on government responsiveness, and which did not? Why?

In answering the research questions, the evaluation should take into consideration the following dynamics:

● The influence of the local geographic, social, economic and political environments on citizen engagement

● The relationships and interactions between the actors involved in program activities, including citizens, community development councils, CSOs, local businesses and local/national government institutions, donors (local and international), etc.

IV. Evaluation Methodology/Contractor Requirement USAID/Serbia expects the assessment to be conducted in two phases.

Phase 1: Research on Programming Approaches

The contract will review available documentation, including but not limited to program plans, project reports, assessments, and mission strategies to develop broad understanding of the methodological approaches of the three programs (CRDA, SLGRP, CSAI), and the degree to which they were able to effectively promote citizen engagement. The contractor must also conduct interviews with key stakeholders based in Belgrade, survey program participants (for example, through an online survey) and gather additional data about the broad sustainability of program interventions. In consultation with USAID, the contractor must also identify case studies for phase 2 of the assessment. The contractor must produce a report summarizing findings from the initial research, indicating how program interventions resulted in sustained citizen engagement following completion of the projects.

Phase 2. Case Studies

The contractor must conduct field research on 8-12 cases that illustrate the projects’ ability to engage citizens in governance processes and ensure continued government responsiveness. Through key stakeholder interviews and other methodologies proposed by the contractor, these case studies must look at how promising examples of citizen engagement succeeded or failed at continuing to engage citizens after the end of USAID support, as well as interventions that led to unexpected long term results. Case studies must also illustrate, confirm, or challenge the findings of the initial research, and provide greater insight into why various approaches have or have not contributed to sustained citizen engagement and/or government responsiveness.

V. Deliverables The following deliverables will be required as a part of the ex-post evaluation.

a. Work plan: A draft work plan to be submitted not later than one week after award of the contract. Adjustments to this work plan will be made based upon comments and/or issues raised by USAID Serbia staff, and USAID will approve the final work plan prior to the start of field work. The work plan must at a minimum include the following:

i. Evaluation methodology, including research approaches and finalized research questions.

ii. A draft schedule of targeted meetings and list of potential interviewees;

iii. Data collection and analysis plan (including interview questions and protocols); and

iv. Identification of potential major constraints.

B. In-brief meeting: The evaluation team must conduct an in-brief meeting with USAID Serbia Mission management subsequent to submission of the draft work plan, in order to discuss the work plan and solicit USAID feedback.

C. Mid-Point Evaluation Meeting

Not less than 30 days after the approval of the work plan, the contractor will have completed the research on programming approaches, and must present preliminary findings to USAID/Serbia staff. This briefing will be based on:

● Review of project reports and evaluations

● Interviews with Belgrade-based stakeholders, including (but not limited to) former CRDA, SLGRP, and CSAI staff;

● Interviews with key legacy institutions based in Belgrade (including but not limited to the

Standing Conference on Towns and Municipalities (SCTM), the National Alliance for Local Economic Development (NALED), and consulting groups formed by former project staff) and other stakeholders as deemed appropriate by the contractor

● Surveys, focus groups, or other methodologies proposed by the contractor.

The briefing must identify broadly the strengths and weaknesses of various approaches employed by past USAID programs in order to engage citizens in governance processes. At the briefing, the contractor must also propose case studies, to be approved by USAID prior to conducting the case studies.

D. Draft Evaluation Report

Following the completion of the case studiesand within 20 days of the mid-evaluation meeting, the contractor must provide a draft evaluation, including research and case studies. The evaluation report must comply with USAID’s Evaluation Policy3 and in particular must meet the following criteria:

- Based on rigorous, objective research methodologies that are described in the research report.

- All data and tools used in conducting the evaluation (questionnaires, checklists, etc…) will be included in an annex to the report.

- Evaluation findings will assess the impact of program interventions on males and females

- Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (including selection bias, recall bias, unobservable difference between comparator groups).

- Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence.

- Sources of information need to be properly identified and listed in an annex.

- The evaluation should provide recommendations for future programming approaches that will result in sustained citizen engagement and government responsiveness

- The report will be provided in English, with sources appropriately cited and minimal grammatical or typographical errors.

3 https://www.usaid.gov/sites/default/files/documents/1868/USAIDEvaluationPolicy.pdf

E. Final Briefing: The team must conduct a final briefing to USAID covering its draft evaluation report, and the findings and recommendations at the conclusion of its field work. The out-briefing will include a rough draft of the evaluation report.

G. Revised Draft Evaluation Report: Within seven days receiving feedback from USAID, the contractor must deliver a revised report, including a four- to-six page executive summary and a full report of up to 30 pages. Additional material may be included as annexes.

H. Final Evaluation Report: The evaluation team must incorporate feedback and provide its final report, with page length specifications as noted for the revised draft evaluation report.

VI. Evaluation Team Composition

USAID/Serbia is seeking applications from locally registered organizations to conduct the ex-post evaluation according to the defined Statement of Work. Applicants should propose a team with the experience and skills necessary to complete this evaluation. The team must include a mix of individuals with:

- At least 10 years of practical experience in evaluation design and social science methodologies

- In depth knowledge of Serbian civil society and the social, political and economic conditions that affect the ability of civil society to represent citizens

- Detailed knowledge of Serbian governance institutions, including government efforts to engage citizens The evaluation team will be responsible for all logistics, translation, and transportation related to conducting the evaluation.