Attachment_to_Amend._1__RFP_Final_Evaluation.pdf

REPORT ON THE MID-TERM

PERFORMANCE EVALUATION

OF THE USAID TRANSPARENCY

AND GOVERNANCE PROJECT

EL SALVADOR

Submitted: 24 December 2012

Data collection: 22 October to 26 November 2012

REPORT ON THE MID-TERM

PERFORMANCE EVALUATION OF

THE USAID TRANSPARENCY AND

GOVERNANCE PROJECT

EL SALVADOR

DECEMBER 24, 2012

Under Contract Number AID-519-O-13-0001

Development and Training Services, Inc.

4600 N. Fairfax Drive, Suite 402

Arlington, VA

USA 22203

Researched and prepared by: Melanie Reimer and Olga Nazario

This report was prepared by Development and Training Services, Inc. The views expressed in this publication do not necessarily reflect the views of the United States Agency for

International Development or the United States Government.

Table of Contents Acknowledgements

Executive Summary

A. Introduction: Project Context and Description

Context

Previous USAID programming

Current Project

Partnership for Growth

B. Evaluation Purpose and Methodology

Evaluation Purpose and Objectives

Evaluation Methodology

General approach

Evaluation questions

Data Collection Methods

Limitations

C. Findings and Conclusions

Most significant performance results

Major factors influencing achievement of objectives

Conclusions

Effects of the implementation approach on results

Conclusions

Involvement and contribution of different sectors of society

Steps taken by the Government to control corruption

Institutionalization and Operationalization by the Government

Conclusions

Awareness of USAID funding among beneficiaries

D. Overall Conclusions

E. Recommendations

Recommendations for Casals in the short-term (during this Project)

Recommendations for USAID in the short-term (during this Project)

Recommendations for USAID in the long-term (following this Project)

Annex A: Evaluation Scope of Work

Annex B: Evaluation Plan and Methodology

Annex C: List of Informants

Annex D: Analysis of Monitoring and Evaluation Indicators

Annex E: Data Collection Tools

Annex F: Matrix of Documents Reviewed

Annex G: Disclosure of Real or Potential Conflict of Interest

Annex H: Supplementary Recommendations for Youth Programming

Table 1: Guide to Evaluation

Questions……………………………………………………………………………………. 11

List of Acronyms AECID Spanish Agency for International Development Cooperation (Spanish)

APES Journalists Association of El Salvador

APIL Law on Access to Public Information

ASI Association of Salvadoran Industrialists

CAM Central America and Mexico

CASALCO Salvadoran Chamber of the Construction Industry

CSO Civil society organization dTS Development and Training Services, Inc.

EMPREPAS Empresarias Salvadoreñas por la Responsabilidad Social

FESPAD Fundación de Estudios para la Aplicación del Derecho

FPEMP Fiscal Policy and Expenditure Management Project (USAID)

FUNDE Fundación Salvadoreña para el Desarrollo

FUSADES Fundación Salvadoreña para el Desarrollo Económico y Social

FY Fiscal Year

GET Government Ethics Tribunal

GOES Government of El Salvador

IACAC Inter-American Convention against Corruption

IAPI Institute on Access to Public Information

IDB Inter-American Development Bank

ISD Iniciativa Social para la Democracia

JCAP Joint Country Action Plan (under Partnership for Growth)

LAPOP Latin American Public Opinion Project

LIDERA Private sector pro-ethics organization that brings together primarily law offices and universities MCC Millennium Challenge Corporation

MCP Municipal Competitiveness Project (USAID)

MESICIC The Mechanism for Follow-Up on the Implementation of the Inter-American

Convention against Corruption

M&E Monitoring and Evaluation

MoP Ministry of Public Works

OAS Organization of American States

OIR Office of Information and Response

PFG Partnership for Growth

PMP Performance Management Plan

SOW Scope of Work

SSTA Under-Secretariat of Transparency and Anti-corruption

TAG Transparency and Governance

TI Transparency International

UCA Universidad Centroamericana "José Simeón Cañas", El Salvador

UAIP Unit for Access to Public Information

UN United Nations

UNDP United Nations Development Programme

US United States of America

USAID United States Agency for International Development

USG United States Government

VMT Vice-Ministry of Transport

Acknowledgements

The dTS evaluation team would like to thank all those who generously contributed their time and shared their information and ideas with us, during the course of this evaluation.

The high level of interest and support from USAID, Casals & Associates, Inc. and all informants – in El Salvador and elsewhere – was above and beyond our expectations, and allowed us to quickly gain a thorough understanding of the project and context. The participation of each and every individual added value to the evaluation, which it is hoped will ultimately contribute to the advancement of transparency and ethics and the fight against corruption in El Salvador.

Executive Summary

Context

The Transparency and Governance Project was designed by the United States Agency for

International Development (USAID) in a context in which there were still very limited openings to publicly discuss corruption and transparency problems, in spite of perceptions of pervasive corruption across society. The election of 2009 brought new opportunities for the Government of El Salvador (GOES) and other stakeholders to address this situation.

Those circumstances provided fertile ground to support significant advances, though progress has become more difficult over time.

The Project

The Transparency and Governance Project (“the Project”) is implemented by Casals &

Associates, Inc. (“Casals”) through a $7.9 million contract (including option years up to

FY2014) focused on controlling corruption in the use of public funds. The contract was signed in May 2009, with a base period that ended in September 2010. The projected end date is March 2014.

The Project has two main objectives: 1) to continue efforts to support various levels of the

Salvadoran government to increase responsiveness and accountability towards its citizens, through increasing transparency in government and ethical behavior by public officials, reducing conditions for corruption in public administration, strengthening the link between citizens and government, and thereby increasing confidence in the system; and 2) to continue to foster civil society participation, particularly the private sector, in promoting transparency, accountability and control of corruption in the use of public funds.

The main client of the Project is the GOES, with a primary focus on the executive branch, particularly the Under-Secretariat of Transparency and Anti-corruption and the Government Ethics Tribunal. The Project has also worked with the private sector, as its secondary target group, and to a lesser extent with civil society. Within the framework of the Partnership for Growth (PFG) signed by the GOES and the United States Government, actions under two institutional strengthening goals (#4 and #6) in relation to the crime and insecurity constraint have been allocated to the Project for implementation.

The Evaluation

The purpose of this mid-term performance evaluation was to help USAID analyze the

Project’s activities, results and approaches to date, in order to use the information for future project design and implementation in the areas of anti-corruption, transparency and governance in El Salvador. The evaluation was carried out by Development and Training

Services, Inc. (dTS) under contract with USAID; it covered implementation from the start of the Project in May 2009 through to September 2012.

The key evaluation questions were defined by USAID as follows:

1. What recommendations can be made to realign or adjust the Project during the remaining time to optimize chances for success?

a. What have been the most significant performance results to date?

b. What were the major factors influencing the achievement or non-achievement of Project objectives?

c. How has the current implementation approach positively and/or negatively affected Project results?

2. To what extent have different sectors of society been involved and contributed to the Project, including: the Government of El Salvador, civil society, the private sector, other international donors and the media?

a. Who should USAID and Casals work with during the remaining time to improve chances of success?

3. To what extent has the GOES taken steps to control corruption during Project implementation?

a. How have anti-corruption and transparency initiatives been institutionalized or operationalized in GOES processes?

4. How aware are beneficiaries that this Project is funded by USAID?

The evaluation was largely qualitative in nature; it included review of over 50 key documents, interviews with 33 key informants, four focus groups with 31 participants, as well as several site visits. Data collection took place in San Salvador and Usulután from late

October until mid-November 2012, followed by a validation workshop with USAID and

Casals staff.

Key Findings

A number of positive or negative factors have influenced the achievement of objectives by the Project, including:

The overall high credibility of the US Government, USAID and Casals in the view of

Salvadoran stakeholders.

The change of government that coincided with both the start of the project and a concerted push for reform from civil society.

Lack of access to updated in-depth analysis of corruption in the country.

High levels of politicization and polarization permeating the society.

Declining political will for tackling corruption.

Low level of interest on the part of the private sector.

Diminished impetus from civil society over time.

The implementation approach of the Project has also had an impact on results, in the following ways:

Innovativeness, respectful relationships, and dedicated and skilled personnel have all been advantages.

Some human resource constraints have been observed, such as lack of ongoing professional development of the staff and over reliance Hiram Morales for numerous public activities.

The Project design focused on the national government and private sector, which had the consequence of minimizing attention to municipal governments and rural areas in general. The design also meant that the traditional civil society organizations were largely sidelined in favor of business-related organizations.

The approach was very flexible, in accordance with the Task Order, which resulted in work plans that were more akin to guidelines, as compared to concrete activity plans, and a regularly changing menu of activities.

The allocation of several Partnership for Growth lines of action early in 2012 has expanded the focus of the Project beyond what was originally contemplated.

Ethics promotion has been a major emphasis with government and other stakeholders; to that end, a wide range of short awareness-raising events has been conducted, of which the results are difficult to determine.

Communication and interaction of the Project with other relevant US Government-funded projects has been minimal; similarly, links with international actors are weak.

The GOES has taken significant strides towards the promotion of transparency, accountability and ethics since 2009, although its direct action on corruption has been less evident. Key advances have included:

Strengthening of the legal framework for anti-corruption

Establishment of the Under-Secretariat of Transparency and Anti-corruption

Establishment of the unified online transparency portal

Systematized accountability reporting by many government institutions.

The government has made variable progress towards institutionalizing and operationalizing initiatives supported by the Project, as follows:

The Government Ethics Tribunal and Ethics Commissions are moderately institutionalized and operational, but only in the capital.

The Offices for Information and Response under the new Law on Access to Public

Information are also moderately institutionalized and operational, though few have been set up beyond the capital.

The Institute for Access to Public Information is not institutionalized and not yet operational.

The Under-Secretariat of Transparency and Anti-corruption is highly operational, and the government accountability processes are moderately operational in the executive branch—but they are only institutionalized in national policy.

Generally, there is a high level of awareness among key Project partners in government, the private sector and civil society that the Project is funded by USAID, which also extends to indirect partners, such as Ethics Commissions, and the international community.

Key Conclusions

Building on the solid foundation and credibility established by the previous USAID transparency and governance program, the current Project has significantly contributed to placing the topics of ethics, anti-corruption, transparency and accountability firmly on the national agenda. In the conducive environment following the election of Mauricio Funes, it has provided invaluable assistance to the GOES in passing and implementing the Law on

Access to Public Information, the new Government Ethics Law and strengthening the

Government Ethics Tribunal; and attempting to instill a culture of ethics among public officials. It has also nudged several professions, including lawyers and journalists, into adopting codes of ethics to guide the behavior of their members, including how they should interact with the GOES.

Despite the initial hope for dramatic changes, the Project has faced many challenges due to the politicization and polarization of most GOES institutions and organizations. The level of political will has not been consistent, resulting in unfulfilled promises such as the failure to create the Institute for Access to Public Information, and a prolonged suspension of the work of the Government Ethics Tribunal. There has been no solution to the on-going politicization of the Court of Accounts, which has closed off an important avenue for enhancing oversight of public administration.

Civil society’s exhaustion from unending controversy over the justice sector and decreasing access to support from USAID and other donors, among other factors, seems to have diminished their capacity and will to advocate for more and faster reforms and engage the population in these efforts. Moreover, the private sector—a traditional partner of previous

Salvadoran governments—has undermined its own capacity to be an effective interlocutor due to ongoing conflicts with the Funes administration, and with some exceptions, shown little genuine interest in advancing transparency and combating corruption. Until there is real commitment from the sector for transparency and anti-corruption initiatives, further substantial investment by the Project is not likely to produce sustainable results.

Nevertheless, achievements have been significant both for the Funes administration and initiatives supported by the Project. Salvadorans interviewed seem confident that they are on the right track to reduce corruption, for example through the establishment of new government institutions and initiatives such as the Offices for Information and Response, which are dedicated to responding to citizens’ demands. The legal framework for anti-corruption, while not complete or perfect yet, has improved; there are new and better laws, regulations and procedures on the books.

El Salvador still has a long way to go to reduce its levels of impunity, strengthen its institutions and ensure the sustainability of these achievements. Achieving optimal results in the promotion of transparency and fight against corruption will only be possible through balanced engagement by USAID and the Project with government and non-government sectors, and through building in-depth capacity in key entities. The time is ripe for the

Project to tighten its focus on consolidation of gains and institutionalization of positive changes, through strategically placed support to the government and civil society during its crucial final year of implementation. Stronger monitoring tools and practices will ensure that the results and lessons of this Project can be captured clearly and capitalized upon in the future.

A major challenge remains—extending the benefits of reforms and new services to the majority of the population, whose daily lives are far removed from the capital and national government, but who continue to suffer the effects of corruption on a regular basis. By prioritizing better coordination and collaboration, and by making anti-corruption a cross-cutting theme of all USAID programming, it will be possible to seek synergies, avoid potential duplication of efforts, and achieve greater and more sustainable impact across the country.

Key Recommendations

1. Recommendations for Casals in the short term

The Project will end in March 2014, which means that activities will be winding down from

December 2013 onwards; that leaves approximately one year for implementation of this set of recommendations.

Establish and cultivate links with other relevant actors and projects, including those funded by US Government and others.

Engage more actively with civil society, to ensure a balanced approach and encourage crucial advocacy for improvements in the legal framework, establishment of the

Institute of Access to Public Information, and other reforms.

Provide strategic assistance to the executive branch of GOES, emphasizing the obligations of both sides, consolidating gains, building in-depth capacity, and ensuring that the roles of key entities such as the Under-Secretariat of Transparency and

Anti-corruption and the Government Ethics Tribunal are kept from overlapping .

Work with the USAID Economic Growth Office and its implementing agencies to identify suitable approaches to engagement with the private sector.

Engage a wide range of relevant experts from the region and beyond to contribute expertise and new ideas, while investing in professional development of staff.

Strengthen monitoring and evaluation within Project and with partners.

Develop activities that will generate visible and tangible short-term wins in the final phase of the Project, to inspire the population and combat impunity.

Support the Under-Secretariat of Governance and Modernization in its efforts to promote a civil service reform.

2. Recommendations for USAID in the short term

Ensure coordination among relevant USAID projects, as a top priority.

Support interaction among the Salvadoran institutions that are key to transparency and anti-corruption in order to improve coordination and resolve areas of apparent overlap of mandates.

Ensure that all USAID projects and key Salvadoran partners are aware of the exact implications of the Partnership for Growth for their work; this is especially important for the private sector.

Update the Performance Management Plan and Project M&E Framework in collaboration with Casals.

Conduct an updated corruption assessment that can support the work of USAID implementers and others.

3. Recommendations for USAID in the long term

Make transparency and anti-corruption a crosscutting theme in the El Salvador mission.

Dedicate resources to transparency and anti-corruption initiatives and entities at municipal level and in rural areas, through both local government and civil society

Simplify M&E systems and indicators.

Consider placing more conditions on assistance and take care not to support institutions known to be corrupt.

Incorporate gender considerations in new programming.

USAID should consider providing more specific anticorruption training to GOES officials and civil society organizations.

A. Introduction: Project Context and Description

Context

The Transparency and Governance Project (“the Project”) subject to this evaluation was designed by the United States Agency for International Development (USAID) in a context in which there were still very limited openings to publicly discuss corruption and transparency problems, in spite of perceptions of pervasive corruption across society. The

2009 Scope of Work described the situation as follows: “The culture of Salvadoran democracy is threatened by unfulfilled expectations for increased economic opportunities, citizens’ overwhelming concerns with violence and crime, the high level of political polarization that impedes public officials from adequately confronting pressing national challenges, and the limited credibility of key democratic institutions and their processes.

There also exists dissatisfaction with the lack of integrity and transparency in public decision makers and processes, particularly in areas related to management of public funds.

Salvadorans continue to cite corruption as one of the main reasons they do not trust their government.” Those difficulties were compounded by a complicated public procurement process, lack of access to public information, non-transparent legislation, weak internal control systems and a highly politicized budget cycle.

The election of 2009 brought new opportunities for the Government of El Salvador (GOES) and other stakeholders to address this situation. During the election campaigns, all presidential candidates publicly committed to a set of transparency and anti-corruption initiatives if elected. After the elections, the Mauricio Funes administration demonstrated early political will to make changes, naming well-known anti-corruption leaders to key positions and creating a Under-Secretariat for Transparency and Anti-corruption. Civil society was also actively engaged in transparency and accountability issues, with some support from the international community. Those circumstances provided fertile ground for the Project to support significant advances, though progress has become more difficult over time. Regional circumstances also complicate the task of fighting corruption as growing transnational organized crime affecting Central America thrives on corruption and inevitably permeates critical institutions, especially in the legislative and judicial sectors.

Previous USAID programming

USAID implemented the USAID/CAM Anti-corruption, Transparency and Accountability

Program from 2004 to 2009 to assist the government, private sector, civil society and other key actors to work together to slow and reverse the spread of corruption in each participating country in the Central American region. From 2006 to 2009, that program supported anti-corruption reforms in El Salvador at the national and local levels, and built advocacy partnerships with some civil society organizations (CSOs). During the program period, the National Assembly passed the Government Ethics Law, and the government established the Government Ethics Tribunal (GET), the Consumer Defense Agency increased its capacity to enforce laws for protecting consumers, and support was provided to development of the government’s report to the Organization of American States (OAS) for the second round of compliance with the Inter-American Convention Against

Corruption (IACAC).

In the justice sector, initiatives were carried out with the Attorney General’s Office to strengthen its Anti-Corruption Unit and with the Public Defender’s Office to enhance internal procedures. As well, the National Council for the Judiciary was assisted to expand ethics training and to develop judicial transparency indicators, and the Supreme Court was supported to develop a code of ethics for judges. At the local level, USAID provided technical assistance to promote decentralization and municipal transparency. Much of the work during that program provided important foundations for the current Project. Activities in El Salvador under both USAID interventions have been implemented by Casals &

Associates, Inc. as lead contractor.

Current Project

The USAID Transparency and Governance Project for El Salvador, under Contract DFD-I-

01-08-00069-00 with Casals & Associates, Inc. (“Casals”) is a $7.9 million project (including option years up to FY2014) focusing on controlling corruption with the Government of El

Salvador (GOES) and civil society in the use of public funds. The contract was signed in May

2009, with a base period that ended in September 2010. The Project has four consecutive option periods; the second option year had just been completed as this evaluation commenced in October 2012.

The Project has two main objectives: 1) to continue efforts to support various levels of the

Salvadoran government to increase responsiveness and accountability towards its citizens, through increasing transparency in government and ethical behavior by public officials, reducing conditions for corruption in public administration, strengthening the link between citizens and government, and thereby increasing confidence in the system; and 2) to continue to foster civil society participation, particularly the private sector, in promoting transparency, accountability and control of corruption in the use of public funds.

The development hypothesis of the Project is: If the Government of El Salvador fights corruption through improved transparency and accountability, then it will not only consolidate the country’s democracy and improve institutional legitimacy, but it will also positively impact El Salvador’s effectiveness to combat crime and insecurity and contribute to economic growth. The main client of the Project is the GOES, with a primary focus on the Executive branch, particularly the Under-Secretariat of Transparency and Anti-corruption (SSTA) and the Government Ethics Tribunal (GET). The Project has also worked with the private sector, as its secondary target group, and to a lesser extent with civil society.

The following three main areas of activity and corresponding sub-activities were defined by

USAID in the Scope of Work of the Project.

Activity 1: Establishing and promoting good governance and anti-corruption practices within the government

Sub-activities:

1.1: Assist the new government to implement transparency, ethics and anti-corruption measures as related to priorities proposed in its government platform

(Plan de Gobierno 2009-2014)

1.2: Continue support to the Government Ethics Tribunal

1.3: Support the GOES to implement the new Transparency and Access to Public

Information Law

1.4: Auditing the Court of Accounts

Activity 2: Engaging Civil Society in the Fight against Corruption

Sub-activities:

2.1: Private sector involvement in the fight against corruption

2.2: Establishment of the Transparency International (TI) Chapter in El Salvador

2.3: Independent Court of Accounts

2.4: New generation leadership

Activity 3: Provide technical assistance and training in support of other openings for relevant transparency initiatives designed to curb corruption

Sub-activities: Have been defined during project implementation

Partnership for Growth

The Partnership for Growth (PFG) Joint Country Action Plan (JCAP) 2011-2015 for El

Salvador and the United States, upon being signed in November 2011, became one of the most important strategy documents governing USAID’s programmatic focus in El Salvador.

The JCAP identified two key constraints to growth: crime and insecurity and low productivity in the tradables sector. Within the framework of the PFG and JCAP, several actions under two institutional strengthening goals in relation to the crime and insecurity constraint have been allocated to the Project for implementation. These actions fall within the overall objectives of the original Task Order of the Project, although some areas of work were not specifically anticipated by that Task Order.

Goal #4: Facilitate economic growth by ensuring El Salvador’s labor force is protected from crime while transiting to and from work, and ensuring that the public transportation service providers serving the labor force are protected from crime. Two specific Lines of Action under this goal correspond to this Project: the USG will support GOES as it reviews the current regulations of the public transportation sector and identifies areas to improve transparency and accountability, and USAID will provide technical assistance to help increase the transparency and accountability of the public transport system.

Goal #6: Professionalize El Salvador’s civil service and enhance public confidence in the government. Two specific Lines of Action under this goal correspond to this Project:

providing technical assistance to promote and facilitate comprehensive civil service reform (with reform of the law being the specific task assigned to the Project) and supporting GOES in increasing responsiveness to accountability towards its citizens through increasing transparency in government and ethical behavior by public officials.

These actions and goals relate directly to USAID/El Salvador’s efforts to promote transparency and governance, particularly through its current Project, but will also direct future programming strategies. Project activity specifically directed towards these two goals commenced in early 2012.1

1 Due to the recent start of these activities, it was premature to attempt to analyze results in relation to those goals. Nevertheless, they have been taken into account by the evaluation, especially in the area of recommendations for future programming.

B. Evaluation Purpose and Methodology

Evaluation Purpose and Objectives

The purpose of this evaluation was to help USAID analyze the Transparency and

Governance Project’s activities, results and approaches to date in order to use the information for future project design and implementation in the areas of transparency and governance in El Salvador. This mid-term performance evaluation (as defined in USAID’s

Evaluation Policy) was carried out by Development and Training Services, Inc. (dTS), whose evaluation team composition is described in Annex B. The evaluation covered implementation from the start of the Project in May 2009 through to September 2012.

Specifically, the objectives of this evaluation were:

To analyze whether USAID is using the right actions and initiatives to control corruption and promote transparency in El Salvador

To assess whether the project has achieved expected results during the base period, as well as option years 1 and 2

To make recommendations on adjustments for optimal implementation during the

Project’s remaining time

To make recommendations on with whom USAID and Casals should work to improve chances of success

To gauge the GOES’ response to the Project in terms of anti-corruption and transparency initiatives

Evaluation Methodology

General approach

This performance evaluation was largely qualitative in nature, and based on extensive document and data review, personal interviews with key informants, focus groups and site visits and observations. Informants from various sectors were identified based on suggestions from USAID, from Project staff, and from other informants, complemented by the personal knowledge of the evaluators. Data collection took place in San Salvador and

Usulután from late October until mid-November 2012.

The original Scope of Work for the evaluation is attached as Annex A, while the full evaluation plan and methodology is at Annex B. In conducting the analyses and reaching conclusions, the evaluators have triangulated findings from documents, interviews, focus groups and site visits. Results were also analyzed in relation to the Performance

Management Plan (PMP) of the USAID Democracy and Governance Office and the M&E

Framework developed for the Project.

Evaluation questions

In accordance with USAID guidelines, the evaluation has focused on descriptive and normative questions, including: (a) what did the Project achieve during the implementation period; (b) how is the Project being implemented? c) how is the Project perceived and valued? More specifically, the evaluation addressed the four major and five associated questions and sub-questions set forth in the Scope of Work (SOW) for the evaluation, which are addressed in the report as set out in Table 1 below.

The Evaluation SOW written by USAID listed the evaluation questions in priority order, starting with the most important question, as directed by USAID’s Evaluation Policy;

however, this order did not follow a logical sequence that translated easily and directly into an evaluation report format. This Evaluation Report presents the findings, conclusions and recommendations corresponding to the evaluation questions in a logical order, with the recommendations at the end of the document. Table 1 is a guide to show how the

Evaluation SOW questions (Column 1) evolved into dTS’ Evaluation Plan methodology

(Column 2 and Annex B), and finally were presented in this Evaluation Report (Column 3).

Table 1. Guide to Evaluation Questions Column 1 Column 2 Column 3

Question # in USAID’s Evaluation

SOW

Question # in dTS’ Evaluation

Plan2

Most Relevant Section of the Evaluation Report

1. What recommendations can be made to realign or adjust the Project during the remaining time to optimize chances for success?

4 Recommendations, page 27

a. What have been the most significant performance results to date?

1 Most significant performance results, page 13

b. What were the major factors influencing the achievement or non-achievement of

Project objectives?

2 Major factors influencing achievement of objectives,

c. How has the current implementation approach positively and/or negatively affected Project results?

3 Effects of the implementation approach on results, page 17

2. To what extent have different sectors of society been involved and contributed to the Project, including: the Government of

El Salvador, civil society, the private sector, other international donors and the media?

5 Involvement and contribution of different sectors of society, page 21

a. Who should USAID and Casals &

Associates work with during the remaining time to improve chances of success?

6 Recommendations, page 27

3. To what extent has the GOES taken steps to control corruption during Project implementation?

7 Steps taken by the GOES to control corruption, page 23

a. How have anti-corruption and transparency initiatives been institutionalized or operationalized in

GOES processes?

8 Institutionalization and operationalization by

Government, page 23

4. How aware are beneficiaries that this

Project is funded by USAID?

10 Awareness of USAID funding among beneficiaries, A table of the methods and data sources relied upon to answer each question can be found in the Evaluation Plan in Annex B.

2 Question #9 in dTS’ Evaluation Plan (How do stakeholders value and/or perceive the Project’s results?) does not correspond to a specific question in the Scope of Work, and it is not addressed separately in this report, although it did help to guide data collection and analysis.

Data Collection Methods

Desk/Document Review: Before arrival in El Salvador, the evaluation team conducted a desk review of relevant Project documents made available by USAID/El

Salvador. The document review then expanded to include other materials identified during the evaluation. In total, over 50 documents were reviewed. All documents subject to substantive review by the evaluators are listed in Annex G, indicating which evaluation question each document was used to address.

Key Informant Interviews: The evaluation team conducted 31 interviews involving 33 individuals (25 male and 8 female), primarily in person as well as several by telephone. These key informants included the 12 organizations mentioned in the

Scope of Work, such as key personnel of the Sub Secretariat of Transparency and

Anti-corruption, Government Ethics Tribunal, Ministry of Public Works, Court of

Accounts, and Attorney-General’s Office, as well as others identified by the evaluation team. Categories of informants were developed to ensure a variety of perspectives; they included USAID/El Salvador, Casals staff and consultants, GOES officials, civil society, media, private sector, international agencies and independent experts. The list of informants is included in Annex C, and interview guides can be found in Annex E.

Focus Groups: The evaluators conducted four focus group discussions with a total of 31 Project stakeholders (19 male and 12 female), such as mayors of small towns and members of an Ethics Commission. The list of focus groups appears in Annex

C. These discussions explored the interaction of the participants with the Project, their views of its activities and results, and their recommendations. Each group had between six and ten participants. Focus group protocols or question guides were tailored to the composition of each group; an example is attached in Annex E.

Site Visits: In order to directly observe the work of the Project and have informal exchanges with relevant stakeholders, the evaluators made six site visits to venues ranging from the “House of Transparency” at the Ministry of Public Works (MoP) to a workshop for civil society representatives. The list of site visits is found in Annex

C. In the case of each site visit, conversations were held with staff or participants at the venue and observations made in relation to how the support of the Project was being utilized.

Validation Workshop: At the close of data collection, the evaluation team organized a workshop to discuss emerging findings and their implications for the Project and future USAID programming. Participants were representatives of Casals and

USAID/El Salvador. This participatory workshop assisted the evaluators in interpreting data, refining their findings, pinpointing gaps and new directions, and assessing the feasibility of recommendations.

Limitations

The evaluation team has experienced the following limiting factors in its data collection and analysis:

Time constraints on data collection, relative to large volume of project documentation and high number of activities over three years.

Flexible nature of the project has involved a number of changes to plans, priorities and activities, which resulted in a complex situation to follow and evaluate.

Lack of concrete data on actual levels of corruption in El Salvador.

Unavailability of key informants from judiciary and media during evaluation.

Project semi-annual reports are activity-focused, and analysis is weak in terms of relating activities to results and results to objectives. Reports do not discuss changes in work plans nor targets for monitoring and evaluation (M&E) indicators.

Little information has been gathered by the Project to monitor results of training and awareness-raising, which has limited the ability to make substantive findings on the quality, relevance and impact of training. Due to contractual limitations and restrictions on how USAID had to report progress on indicators to Washington, DC, some indicators and targets in the official PMP and the M&E Framework have not been updated since the beginning of the Project. As a result, some “actual” data on indicators bears little relation to the “target” numbers, which has constrained the evaluation’s use of indicator data in analyzing yearly and overall results.

C. Findings and Conclusions

In this section, responses are presented to seven of the nine evaluation questions, each in a separate sub-section. Where both findings and conclusions were required to respond to a particular question, the conclusions are presented separately, following the findings for that question. The two remaining questions are answered in the Recommendations section on page 27.

Most significant performance results

Since its commencement in May 2009, the Transparency and Governance Project has contributed to the achievement of a range of results. These include increased transparency and accountability awareness among various Salvadoran target groups3, establishment of new mechanisms and procedures, and improvements in the anti-corruption legal framework of the country.

The Project increased levels of awareness and openness among Salvadoran citizens and public officials about transparency and anti-corruption and helped place these topics firmly on the public agenda and in the mass media. This awareness is especially prevalent among urban residents, according to informants, as compared to rural citizens with less access to information. Recognition of the importance of transparency and ethics has been heightened among several key professions, including journalists and lawyers, through capacity building and development of codes of ethics. The Project also boosted awareness of ethics and transparency among certain key justice institutions, including through a draft code of ethics for the Judiciary and draft Law on Probity.4

In addition, civil society organizations and public officials have greater awareness of the responsibility of government entities to be actively accountable to the public for their actions and spending. Most executive institutions are now reporting publicly, with various degrees of efficacy, on their activities. A noted case is that of the construction and public works sector, widely seen as one of the most corrupt in the country, which has shown increased awareness and will to tackle corruption through engagement with the Project.

3 There is no quantitative survey data on the specific levels of awareness or capacity mentioned in this section, so the evaluation has relied on the views of informants and documentary review in making these findings.

4 Work of the Project with the justice sector was suspended by USAID in order to avoid overlap with another project focused on rule of law, so these initiatives were never finalized.

In terms of the legal framework for transparency and anti-corruption, the Project contributed to passage of the Law on Access to Public Information (APIL) through support to drafting and advocacy by civil society and other activities, which began under the predecessor program. Subsequently, implementation of the APIL has advanced considerably with the support of the Project, primarily through the Under-Secretariat of Transparency and Anti-corruption (SSTA). Important steps have included establishment of at least 80

Information and Response Offices (OIRs), dissemination and socialization of the law, and two public reports on compliance with the law.

The work with the SSTA, which has played a key role in advancing the implementation of several laws and mechanisms, has generally contributed to increased availability of and access to public information. The standardization of executive branch websites with official information and creation of the unified government transparency portal were major milestones achieved with Project support. Most recently, the Project has lent support to creation by SSTA of the Virtual Government Training Program, which aims to extend on-line training opportunities in ethics, transparency, anti-corruption and related topics to a significant portion of public servants in the country.

The knowledge base about the APIL, access to public information in general and transparency has been enhanced through establishment and implementation of a 4-month diploma course at a prominent national university. This course, offered at the Universidad

Centroamericana (UCA), has primarily targeted central government officials (especially

Public Information Officers) to date but also included some civil society representatives.

The Project has built capacity of the Government Ethics Tribunal (GET) and the Ethics

Commissions in various central government institutions through training, technical assistance, and development of tools and procedures, among other interventions. Reform of the Law on Government Ethics in early 2012, which has strengthened the legal framework considerably, was an important achievement supported by the Project. The new law gives more authority to the GET to pursue cases on its own initiative and allows for anonymous complaints and fines up to nearly US$9000.

Transparency in public works procurement has been enhanced through support to development of 26 integrity pacts and other tools, reprinting of the construction industry code of ethics, facilitation of collaborative relations between the industry and the Ministry of

Public Works, and an observatory to monitor public works projects.

In addition, the Project has made a contribution to increasing the capacity and profile of civil society organizations in order to be credible interlocutors with government and influence legislation and policy, building on the strong foundations laid by the previous program.

Work in this area included support to the Foundation for National Development (FUNDE) becoming the national chapter of Transparency International.

Major factors influencing achievement of objectives

A number of factors had a major influence on the Project’s ability to achieve its objectives, including the significant results outlined above. The credibility of the United States

Government (USG) and of USAID among key stakeholders and the public of El Salvador was remarked upon by various informants. USAID is well known and respected as a leading proponent of transparency and accountability initiatives.

As well, the previous transparency and governance program had laid a strong foundation for the current Project. Casals and its Chief of Party had already developed strong relationships with key actors, an understanding of the political and cultural context of El Salvador, and a high level of credibility. The Chief of Party is widely recognized as a diplomatic interlocutor who is able to deal with individuals and entities of all political inclinations in a balanced way and skillfully facilitate cooperation even among diverse stakeholders.

The election of a new government shortly before the Project commenced paved the way for certain legislative and other reforms, especially given the initial political will for transparency demonstrated by President Mauricio Funes. The change of government coincided with a strong push from a core group of civil society organizations (Grupo Promotor) that already had relevant capacity and had been advocating for a law on access to information and other transparency measures. The new government appointed certain individuals known as

Analysis of Results according to PMP and M&E Framework

The outputs and results of the Project were also analyzed by the evaluation in relation to the PMP of the USAID/El Salvador Democracy and Governance Office and the more detailed Project M&E Framework. The two indicators on the Strategic Objective level relate to the Corruption Perception Index published by Transparency International and the level of perceived government responsiveness as measured by the biennial surveys of the Latin American Public Opinion Project

(LAPOP).

With respect to the Index, El Salvador has seen a general downward trend in perception of corruption since 2005, but it has been stable with slight fluctuations from 2009 to 2011. A minor improvement was observed between 2011 and 2012, as the rating climbed from 3.4 to 3.8 (with 10 being the least corrupt). The target set for 2012 was 3.4. As for government responsiveness, the combined proportion of Salvadorans who consider the government to be either “a little” or “a lot” responsive has been fluctuating between 34% and 39% since 2004 baseline, but there is no new data since 2010. It is not possible to draw any conclusions regarding the impact of the Project on these indicators.

With respect to activities focused on the government, the indicator data shows that the government has taken various positive steps showing commitment, that about 10,000 participants have been reached by training activities. As for activities with civil society and private sector, key results captured by the indicators are the passage of the APIL, over 10,000 participants in training and awareness raising, and various initiatives and mechanisms supported, though measurement has proven difficult due to unclear definitions. The majority of targets have been met or exceeded by reported results of the Project, although some targets were not updated over time. No indicators were established for Activity 3.

A detailed analysis by the evaluation of progress achieved against indicators and measurement challenges is available in Annex D.

champions of transparency (for example, Minister of Public Works Gerson Martinez and

Undersecretary of Transparency and Anti-corruption Marcos Rodriguez) to influential positions, which they have used to push for greater transparency and ethics and to combat corruption.

There were other factors, primarily related to the context of the Project, which have hampered or limited the achievement of objectives. The evaluation team found that the base of in-depth knowledge and analysis of the characteristics of corruption in the country was not optimal; the most recent comprehensive corruption assessment conducted by USAID in

2010 was not available to the Project because the final version was never approved for distribution.

Generally, the high level of politicization and polarization that permeates the country makes it difficult to reach compromises and get bipartite or multipartite support for initiatives and key appointments. This is especially true in the case of politically sensitive reforms, such as transparency and anti-corruption actions. The evaluation observed a declining level of high-level political will in this area, as evidenced by the long delay in appointment of commissioners to GET following the law reform; failure to establish the Institute for Access to Public Information (IAIP) as mandated by the law; failure to appoint a new Attorney

General; and inadequate budgets for transparency and anti-corruption entities including the

SSTA, GET and IAIP.

The continued inefficacy of the Court of Accounts, the body primarily responsible for oversight of the public administration, detracts generally from the credibility of anti-corruption initiatives and limits the key institutions with which the Project can interact effectively. As well, progress has been constrained by the lack of a professional and highly qualified public administration staff with job security, which could provide continuity and sustainability to the Project’s initiatives.

Informants have observed a reduced level of pressure by some civil society…