ATTACHMENT 13 - ASSESSMENT AND IMPACT EVALUATION OF COLOMBIA JUSTICE HOUSE PROGRAM.pdf

MARCH 2012

This publication was produced for review by the United States Agency for International Development. It was prepared by Management Sciences for Development, Inc and subcontractor, Partners Colombia.

ACCESS TO JUSTICE PROGRAM

ASSESSMENT AND IMPACT EVALUATION OF

COLOMBIA JUSTICE HOUSE PROGRAM

ACCESS TO JUSTICE

PROGRAM

ASSESSMENT AND IMPACT EVALUATION OF

MARCH 2012

DISCLAIMER

This report was produced by Management Sciences for Development and Partners Colombia, Inc. under the Access to Justice Program, Contract No. AID-DFD-I-00-04-00175, for review by the U.S. Agency for International Development (USAID). The author‘s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United

States Government.

Acknowledgements

This report was prepared by Partners Colombia under subcontract to Management Sciences for

Development Inc. We extend special recognition to Dr. Linn Hammergren, MSD consultant, for her guidance and leading the Partners Assessment Team in organizing, drafting, defining and editing the report.

Partners Colombia Assessment Team

Gustavo Salazar Arbelaez, Coordinator and Technical Supervisor

Core Technical Team

Edgar Ricardo Serrano, Senior

Researcher

Jaidivi Núñez Varón, Senior

Researcher

Heyder Alfonso Camelo, Junior

Researcher

Carlos Alberto Bula, Financial

Consultant

Edwin Girón, Statistician

Olga Cecilia Pinilla, Research

Advisor

Victoria Gaitan Pinilla, Assistant

Researcher

Field Professionals

Jan Carlos Campo

María Cristina Zapata

Johana Paola Romo

Alejandro Quintero Salazar

Ana María Ospina

Wenseslao Villa Rivera

María Constanza Amaya

Ela Lucia Padilla

Raúl David Rangel

Yaneidy Bayona Tarazona

Josue Arnoldo Moreno

Tania Alejandra Pico

Regional Team

Paola Alejandra

Martínez (Bajo Cauca)

Patricia Elena Ruiz

(Montes de María and

Barranquilla)

Ivan Darío Benavides

(Nariño)

Edna Reynet Hurtado

(Meta)

MSD Review, Editing and Graphics

Michael Willis, former Chief of Party for Access to Justice Program (AJP)

Javier Said, Chief of Party for AJP

César Castillo, Access to Justice Advisor

Ana Daza, Regional AJP Coordinator

Angel Cardona, Chief Operating Officer

Yeree Woo, Business Development Associate

Audry Tafoya, Proposal Management and Business Development Coordinator

The assessment team could not have done its work without the cooperation of officials from

Colombia‘s Ministry of Justice, the National Department of Planning, the authorities of the Department of Antioquia, and the Secretary of Government of the Capital District of Bogotá. It would like to express its gratitude for the help they provided. The team also wishes to acknowledge the support received from the group of experts and persons interviewed who provided insights and documents to enrich the report‘s contents (Annex 4).

The team expresses its appreciation to the staff of the twenty-one Justice Houses visited and to the staff members and other individuals who participated in the surveys and focus groups. They are too numerous to mention by name, but without their cooperation none of this would have been possible.

ASSESSMENT AND IMPACT EVALUATION OF COLOMBIA‘S JUSTICE HOUSE PROGRAM

Table of Contents

Executive Summary

Introduction

Chapter I: Background on the Justice House Program

Setting

Models and Normative Framework

Growth and Staffing of the Program

Conclusions

Chapter II: The Demand Side: Findings on Justice House Services, From the User

Perspective

Introduction

Who Uses the Services of the Justice Houses?

Why Do They Use the Justice Houses?

To Whom Do Users Go for Services and What Are the Results?

Did the Justice House Experience Remove the Hypothesized Barriers to Justice?

Did Different Groups Evaluate the Experience Differently?

Additional Issues and Questions Covered

Conclusions on Demand and on User‘s Response to Services

Chapter III: The Supply Side- Finding on Justice House Operations

Introduction

Personnel

Infrastructure, Equipment, Maintenance, and Location

Program Administration

Conclusions

Chapter IV: Comparing Supply and Demand: What Works What Doesn‟t Work and How

Can We Explain the Differences

Introduction

Some Problematic Areas for Justice Houses

Adequacy of Services Provided to Rural and CSDI Populations

Performance Differences among Justice Houses

Different Visions, Different Conclusions

Chapter V: Reflections and Recommendations

Introduction

Short-Term Recommendations that Do Not Depend on Answers to the Bigger Questions

The Bigger Issues that Need to be Addressed as a Prelude to Medium and Long-Term Planning

Medium-Term Alternatives as a Prelude to Longer-Term Development Conclusions

Conclusions

Bibliography

LIST OF ANNEXES

Annex 1. Methodology

Annex 2. Infrastructure Table

Annex 3. Questionnaire

Annex 4. List of Persons Interviewed

Index of Graphs and Tables

INDEX OF GRAPHS AND TABLES IN EXECUTIVE SUMMARY

Classification of Municipios where the 21 Justice Houses are located

Justice House Users and Non-Users by Gender, Ethnic, and Other Categories

Comparison of agreements and level of compliance, by type of dispute, CSDI status (Poll only)

INDEX OF GRAPHS

Graph 1: National and Local Entities Which May/Should be Part of Each Justice House

Graph 2: Rural-Urban Division by CSDI and Non-CSDI Justice Houses, Villavicencio and Villa de

Socorro

Graph 3: Offices and services accessed by service users in the Justice Houses

Graph 4: Response to question ―Would you return to the Justice House?‖

Graph 5: Response to the question ―Would you recommend the Justice House to others?‖

Graph 6: Users‘ Confidence in Entity Providing Service

Graph 7: User Confidence in House as a Whole

Graph 8A: Response to question, ―Who did citizens go to before the arrival of the House?‖

Graph 8B: Response to question: ―Why do citizens now use the Justice Houses?‖

Graph 9A and 9B: Percentage of staff reporting cases unattended and the type of unattended cases

INDEX OF TABLES

Table 1: Number of Justice Houses and Users, 2002-2010

Table 2: Staffing, 2002-2010, all Justice Houses

Table 3: Classification of Municipios where the 21 Justice Houses are located

Table 4: Justice House Users and Non-Users by Gender, Ethnic, and Other Categories

Table 5: Socio-economic classification of Users by Justice House

(Poll Only, Asterisk denotes CSDI)

Table 6: Types of Services Requested of Justice Houses, Villavicencio, Villa de Socorro and Poll

Table 7: Types of requests made to all Justice Houses, 2002-2009 (Census)

Table 8: Comparison of agreements and level of compliance, by type of dispute, and CSDI status

(poll only)

Table 9: Answer to Question ―How did negotiation affect your relationship with the other party?‖

Table 10: Response to the question ―Did the time required to ‗get access to justice‘ decrease?‖

Table 11: Response to the question ―Did the cost of getting ‗access to justice‖ decrease?‖

Table 12: Response to the question ―Did the authorities resolve the problems more rapidly?‖

Table 13: Response to the question ―Does going to authorities bring more problems?‖

Table 14: Response to Statement ―I now understand the authorities better.‖

Table 15: Response to Statement: ―The community has learned to resolved its problems in a better fashion

Table16: Women‘s Evaluation of Justice Houses

Table 17: Afro-Colombian Evaluation of Justice Houses

Table18: Indigenous Evaluation of Justice Houses

Table 19: Responses from focus group women on their assessment of the Justice House services

Table 20: Focus Groups, Afro-Colombian men‘s response to the question, ―What would happen without the Justice House?‖

Table 21: Perceptions of Discriminatory Treatment

Table 22: Answers from focus group (mestiza women) to question ―How were conflicts resolved before and how are they resolved now?‖

Table 23: Answer to question ―With the Justice House is the community more secure?‖

Table 24: For 21 Justice Houses covered in the study, professional staffing 2009

Table 25: Staffing Patterns in 21 Justice Houses, 2010, by Office and Number of Staff in Each

Table 26: Services provided to interviewees in the 21 Justice Houses

Table 27: Evolution of NJHP by phases and types of municipios receiving Justice House

Table 28: Differences in responses from rural and urban populations to the four ―satisfaction questions.‖

Table 29: Response to Question ―How did you feel after going to the Justice House?‖ by CSDI and non-CSDI focus group participants

Table 30: Percentage of Positive Scores and Answers to Selected Questions for individual Justice

Houses, arranged by category of municipio

List of Acronyms and Abbreviations

ADR Alternative Dispute Resolution

CADE Specialized District Attention Centers

CAIVAS Center for Integrated Attention to Victims of Sexual Violence

CAVIF Center for Attention to Intra-familial Violence

CRIs Information Reception Centers

CSDI Colombia Strategic Development Initiative

CSO Civil Society Organization

DANE Departamento Administrativo Nacional de Estadística

FES Higher Education Foundation(Fundación de Educación Superior)

GOC Government of Colombia

ICBF Colombian Institute of Family Welfare

ICT Information and Communication Technology

IT Information Technology

LGBTI Lesbian, Gay, Bisexual, Transgender and Intersex

MASC Alternative Dispute Resolution Mechanisms

MIJ Ministry of Interior and Justice

MSD Management Sciences for Development

NGO Non-Governmental Organization

NJHP National Justice House Program

SAU Office for Attention to the User

SENA National Job Training Service

SISBEN Subsidized Health Program

UAI Unidad de Asistencia Integral

URI Immediate Attention Center

USAID United States Agency for International Development

ASSESSMENT AND IMPACT EVALUATION OF COLOMBIA‘S JUSTICE HOUSE PROGRAM 1

Executive Summary

INTRODUCTION

This assessment of Colombia‘s National Justice House Program (NJHP) was done for USAID/Colombia by MSD and its subcontractor, Partners/Colombia. The assessment comprised a literature review and extensive fieldwork on the operations of 21 of the 73 Justice Houses in existence in early 2011. It focuses on evaluating the program‘s performance, identifying problems and their causes, and forwarding recommendations to address them, especially as regards:

The program‘s legal framework and the adequacy of administrative, logistical and material support

The program‘s success in fulfilling its strategic objectives, including reduction of barriers to citizen access to justice; enhancing state presence at the local level; reduction of judicial backlog through use of alternative dispute resolution mechanisms; and development of programs to prevent domestic violence

The program‘s ability to resolve the legal needs of vulnerable populations in particular, as well as those in conflict zones and rural areas

Staff strengths and weaknesses especially as regards assimilation of skills, abilities and knowledge to conduct the program

The fieldwork was conducted by Partners and utilized a mixed methodology. Its most important element was a series of structured interviews with 5,646 service users and 746 non-users from the 21 Houses, drawn from ―non-random‖ samples in 19 Justice Houses (collectively called the sondeo, translated as

―poll‖ throughout the document)1 and random samples in two. They were complemented by 35 focus groups, interviews with 84 staff members, and the observations of staff conducting the fieldwork.

NJHP BACKGROUND AND LEGAL FRAMEWORK

The NJHP originated in the early 1990s, inspired by Colombia‘s rights-rich new Constitution (1991), persisting weaknesses in its formal justice system, and non-incidentally studies done by USAID consultants to develop proposals for improving access to justice for socially and geographically marginalized groups. It is not the only such access enhancing program in the Latin American region or in

Colombia, but as created and further developed it has the greatest potential for bringing legal and related services to poor, rural populations and also has accumulated a unique set of additional objectives over its 17 years of existence.

The NJHP entered official existence with its inclusion in the Justice Development Plan for 1994-1998 under the title ―Justicia para la Gente‖ (Justice for the People). One year later (1995) the first Inter-

Institutional Agreement was signed, setting out the program‘s objectives and committing various national and local government agencies to participation, and the first two Houses were established. The five objectives given in the agreement stressed legal information and orientation for community members, and, significantly, did not emphasize alternative dispute resolution (ADR), although it would become the

1 There were contextual problems (lack of adequate registries) which prevented the construction of random samples. The assessment team attempted to compensate by using registries where available and randomly selected individuals exiting the Houses, but we admit these were not ―probabilistic samples.‖

ASSESSMENT AND IMPACT EVALUATION OF COLOMBIA‘S JUSTICE HOUSE PROGRAM 2

program‘s major thrust and already was an interest of the Ministry of Justice, the agency entrusted with its coordination.

By 1997, with the appointment of a new Minister and the replacement of most staff in the Access

Division where the program was located, dispute resolution had become the central theme, and there was a move to extricate services that did not support it.2 In 2000, a series of critical studies precipitated another change and the enactment of Decree 1477/00, which to this day is the only law regulating the program and thus the most formal statement of its objectives. However, while the Decree added programs for families and minors and to promote the defense of human rights and combat abuses, it made participation by the inter-institutional partners voluntary. A final significant change came with the

NJHP‘s linkage to Colombia‘s Consolidation Plan3 in the mid-2000‘s. Another Inter-Institutional

Agreement signed in 2005, while changing neither the purpose nor the voluntary participation of national agencies, did attempt to establish responsibility for recurrent costs, giving most of them to the local governments. It also mandated the creation of information reception centers (CRIs) in each Justice

House, and created three new models – the Justice House, the Center of Convivencia (to be established in municipios with fewer than 100,000 inhabitants), and an ―articulated model‖ as yet to be defined.4

Three general conclusions about the legal framework follow. While this is an inter-institutional program, there is still only one law regulating it, and it does not compel participation. The issue of the program‘s purpose and content has never been resolved. The creation of new Justice Houses and the operations of those in existence have honored legal requirements and ―suggestions‖ in the breach, so that what exists is largely a function of what local authorities wish to finance or can negotiate on their own with national agencies. This is particularly a problem for smaller municipios, those emphasized under the Consolidation

Plan many, but not all of which, are included in the U.S. Embassy/Bogotá‘s Colombian Strategic

Development Initiative [and called CSDI] regions by USAID, as they rarely have the funds to implement the complement of services and entities mentioned by Decree 1477/00. Nonetheless the program has grown to include 77 Justice Houses (only 73 at the time of the fieldwork) and another 30 or so Centers of Convivencia.

THE DEMAND SIDE

Pursuant to consultations with USAID, Justices Houses covered in this review were selected by

MSD/Partners to include those from all sizes of municipios,5 a mix of CSDI and non-CSDI regions, and

Houses that had existed for longer, moderate and relatively short periods of time. An effort was also made to include Justice Houses that draw, at least in theory, on rural as well as urban populations.

Municipios are legally classified by size (and average income), ranging from the Special category, with over

500,000 inhabitants to the sixth category with 10,000 or less.

2 The Access Division was this office's title during the entire period covered by this report. However, its name was recently changed to "MASC Division" or Division for the Alternative Resolution of Conflicts; this title is not used here in part for historical reasons, but also because with the name change, it is still unclear as to how the office's functions and organization will be affected.

3The GOC‘s National Consolidation Plan seeks to consolidate security and social gains in 51 municipalities by building state presence in previously ungoverned areas of the country, and by providing security, law enforcement, rule of law, and development assistance in a coordinatedand combined manner.The US Embassy‘s inter-agency

Colombia Strategic Development Initiative aims to complement and support the National Consolidation Plan by focusing US assistance in strategic areas known as ―CSDI regions.‖ 4 The Centers are not covered in this report, which only focuses on Justice Houses.

5 The municipio is the lowest level administrative unit in Colombia, roughly equivalent to a county, although it may be a part of a larger city or a large, very rural district with several small population centers.

ASSESSMENT AND IMPACT EVALUATION OF COLOMBIA‘S JUSTICE HOUSE PROGRAM 3

Classification and CSDI Status of Municipios where the 21 Justice Houses are located

URBAN RURAL

City/Department Location Category/CSDI

Status

Department/

Región

Location Category/CSDI

Status

Bogotá, D.C. Usme Special/No Cauca Puerto

Tejada Sixth/No

Cali Aguablanca Special/No Bajo Cauca Tarazá

Sixth/CSDI for all Cundinamarca Soacha Secondary/No El Bagre

Antioquia Bello Special/No Zaragoza

Meta Villavicencio Secondary/”CSDI”

Sur del Tolima

Ataco

Santander Bucaramanga Special/No Chaparral

Barranquilla La Paz

Primary/No

Rioblanco

Simón Bolívar

Medellín Villa del

Socorro

Special/‟No Ortega

Cauca Popayán Tertiary/No

Guajira Riohacha Fourth/No

Nariño Pasto Secondary/CSDI

Nariño Tumaco Fourth/CSDI

Who uses the Justice Houses?

This question is best answered by the table below, based on surveys done by the assessment team:

Justice House Users and Non-Users by Gender, Ethnic, and Other Categories Characteristics Poll

N=2,416

Villavicencio

N=1,159

Villa de

Socorro

N=1,143

Non Users

N=586 (Poll only)

No % No % No % No %

Women 1574 65.2 766 66.1 846 74 314 52.7

Men 837 34.64 393 33.9 297 26 278 46.6

LGBTI 5 0.16 0 0 0 0 4 0.7

Afro- Colombians 583 24.15 8 0.7 ND ND 105 17.7

Indigenous 74 3.1 4 0.35 4 0.35 23 3.9

Socio-economic status 1 1.331 55.1 792 68.34 110 9.62 242 40.6

Socio-economic status 2 864 35.8 319 27.52 996 87.1 247 41.4

Socio-economic status 3-5

(for non users 3-6) 177 7.3 30 2.6 12 1.1 101 17

Socio-economic status

DK/NA*

45 1.85 18 1.55 25 2.2 6 1%

ND: The ethnic classification for Medellin (Villa de Socorro) is not registered because most respondents simply categorized themselves as “paisas” – residents of the department of Antioquia. Entries for the poll are weighted to give equal value to each of the 16 separate samples.

6 There was a discrepancy here between Puerto Tejada‘s classification according to Law 617/00 and that used by

DANE (Departamento Administrativo Nacional de Estadística). Under the former it should be a category 4 municipio, but DANE considers it in category 6. The assessment team used the DANE classification.

ASSESSMENT AND IMPACT EVALUATION OF COLOMBIA‘S JUSTICE HOUSE PROGRAM 4

DK/NA= Don‟t Know/ Not Applicable

Source: Assessment Team’s data and analysis

Users are predominately women, and nearly exclusively from the lowest socio-economic status categories (1 and 2). Afro-Colombians make up about a quarter of the responses to the poll, and probably would do the same for Villa del Socorro except that respondents Medellin call themselves paisas (residents of Antioquia department). In contrast, non-users are less predominately women and also come from higher socio-economic status. In short, it appears that the Houses are attracting the intended target population, taking into account variations among the municipios where each is located.

The only missing element is rural. Rural users constituted only 10.6 percent of those in CSDI regions, less than 5 percent in non-CSDI regions, and 2.25 and 0.7 percent in Villavicencio and Villa del Socorro, respectively. There are several possible explanations: the older Houses tended to be built in urban areas; those in CSDI regions and thus largely rural regions are late arrivals (and so even 10 percent may be a ―success‖), and in these same areas transportation and communication remain very difficult, making even the suggested use of itinerant Justice Houses somewhat impractical. Sometimes people live too far away to be reached, but this is a problem both from the standpoint of justice and from the government‘s interest in ―establishing state presence,‖ a goal added under the Consolidation Plan.

Why do they use the Justice Houses?

The simple answer here is: roughly 50 percent to resolve disputes, with family matters being the predominant type, and the rest for a variety of services, with the most important being obtaining or replacing documents, other civil and criminal matters, requests for information, and in the CSDI regions and Villavicencio, training. The relative importance of dispute resolution and of the types of disputes varies among Houses and by CSDI and non-CSDI regions, but the important point is that while dispute resolution is the most frequent service, there are many others.

What are the results?

The survey focused on results in dispute mediation. The most significant findings are shown below:

Comparison of agreements and level of compliance, by type of dispute, CSDI status (Poll only)

Nature of Dispute

NON-CSDI

ALL DISPUTES =978

CSDI

ALL DISPUTES = 464

VILLA DEL

SOCORRO

ALL DISPUTES=853

VILLAVICENCIO

ALL DISPUTES=727

% With

Agreement

Agreement

Carried

Out

% With

Agreement

Agreement carried out

% With

Agreement

Agreement carried Out

% With

Agreement

Agreement

Carried

Out

Family 89.52 69.44 67.07 64.24 89.13 75.61 82.07 70.46

Criminal 71.68 61.73 72.55 56.76 82.29 69.62 76.58 85.95

Other 56.67 52.94 62.07 44.44 84.15 79.71 100 50

Landlord-tenant 86.49 54.69 60 66.67 100 62.5 66.67 77.27

Other Civil 75 65.43 62.26 51.52 86.84 72.73 81.33 77.05

Disputes between neighbors 78.21 62.3 81.54 75.47 83.67 73.17 90.48 87.72

Total 83.64 66.26 68.53 62.58 87.1 74.56 80.88 76.19

Figures for CSDI and non-CSDI are weighted samples Source: Assessment Team’s data and analysis

ASSESSMENT AND IMPACT EVALUATION OF COLOMBIA‘S JUSTICE HOUSE PROGRAM 5

While there are differences among the four samples (with the CSDI region having the lowest scores), the percentage of disputes reaching agreements and of agreements being carried out is extraordinarily high, so much so as to raise questions about respondents understanding what they were being asked. In response to another question on results, as to how it affected post-negotiation relationships, between

21 and 49 percent said they improved, and between 14 and 20 percent said there was no difference.

More importantly, only 3-10 percent said they worsened. Hence, with the small note of skepticism about success rates, the Justice Houses score well on their primary role of resolving disputes.

The scores are even higher on what we have called the four ―satisfactions questions‖ – Would you return to the Justice House? Would you recommend it to others? Do you trust the entity delivering services? Do you trust the Justice House? Positive answers here were all at the 85 to 95 percent level with the CSDI regions scoring nearly as high as the others.

Did the Justice Houses remove the hypothesized barriers to access?

Here eleven questions were asked, six having to do with physical and other concrete barriers (cost, time) and the rest to address cognitive barriers. There was a problem with the formulation of the questionnaire, which meant we only counted positive answers (things improved – but then again there was no reason to believe they would worsen).7 Answers about reduction of barriers showed a much greater range, with CSDI Houses nearly always getting lower scores, but 42 percent saying access took less time or 57 percent saying going to the authorities did not bring problems, even if the non-CSDI regions and the two other Houses scored 10 to 20 points higher. The more important finding is instead that despite lower scores on barrier reduction, for all, the value attached to the Justice House experience was universally extremely high. This suggests, in line with part of the Consolidation Plan strategy, that presence and demand satisfaction have a value of their own, quite independent of what it may take to access services.

Do different groups evaluate the experience differently?

The interest here was in ascertaining whether so-called vulnerable groups – women, Afro-Colombians, indigenous populations – were equally satisfied with the experience. All in all, the survey responses suggested that the scores on the four ―satisfaction questions‖ were as high as or higher than those in the general population. The focus groups showed slightly different results. Mestiza women gave the

Houses equally good scores, and indigenous women, a very small group, rated them all ―excellent.‖

Afro-Colombian women and men made more complaints, criticizing the Houses for excessive bureaucracy, lack of attention to community practices, and sometimes just for poor service. The focus groups for the latter were very small (26 members in all) and hence the results cannot be given too much significance. Still, they may be signs of problems to come and thus merit further investigation.

On the topic of ―differential treatment‖ for the same groups (a Colombian concept, backed by the

Constitution8), it does not appear that discrimination was a problem. Few members of any of these groups said the treatment was anything but ―respectful‖ or ―normal.‖ We are taking ―normal‖ as positive as we doubt many of those responding saw it as anything but that.

Other objectives pursued by the program

7 The problem was two ―no change‖ categories – things remained the same and they did not change. However, the absence of a real negative is probably not important as complementary material from the focus groups did not suggest the Houses made things worse in any of the categories.

8 Pursuant to Article 13 of the Constitution the state will adopt means to ensure marginalized groups have real and effective equality before the law.

ASSESSMENT AND IMPACT EVALUATION OF COLOMBIA‘S JUSTICE HOUSE PROGRAM 6

These included increasing the legitimacy of the state, enhancing security, and reducing court congestion.

The fieldwork tapped only one of these directly – security – and then not in the sense intended by

USAID which had more to do with the security of the Houses. The issue of state legitimacy rests on the debatable hypothesis that providing a valued service (access to dispute resolution and related services) will enhance it. That may be true especially for populations who had no services of any kind before, but as in these same areas, most services are local in origin, what may be enhanced is the legitimacy of local authorities. Impacts on court congestion are even more debatable as the access is being provided to people who would not have gone to court in the first place. There are some additional details which might further decrease or enhance congestion but all and all this seems to be a questionable hypothesis. As for security, the survey did ask users to evaluate how their own security or that of their community had changed and found positive perceptions, ranging from 35 percent in CSDI regions to 68 in Villa del Socorro. Our answers to the USAID concern about Justice House security are largely impressionistic. First, these Houses are nearly by definition located in less safe zones, and there is little that can be done about that. Second, while there have been attacks on Justice Houses only two of them seemed to target the House as any more than another government building and those two were motivated by some specific errors unlikely to be repeated.

THE SUPPLY SIDE OF JUSTICE – FINDINGS ON JUSTICE HOUSE

OPERATIONS

Personnel

Most Justice House personnel are local government employees. Despite the various inter-institutional agreements, the deployment of personnel from national agencies is only common in the larger and more urban Justice Houses. Total staffing among the 21 Houses surveyed ranged from 5 to 1189 (see Table

25), and in the larger Houses many staff members are part-time, voluntary workers in the conciliation centers and legal clinics. Smaller local governments often have to use permanent staff on a part-time basis or contract people to fill the necessary slots. This is frequently the case with coordinators, Information Technology (IT) staff, and those working in the CRIs. However, both there and in the larger

Houses, representatives of national agencies often work part-time and on unpredictable schedules.

Stretched agency budgets and creation of their own multi-service centers contribute to these trends.

Other personnel problems include frequent turnover, limited training, and insufficient coordination, especially but not exclusively between local and national staff.

Infrastructure, equipment, maintenance, and location

Most of this, including the location if not the construction of the Justice Houses, depends on local authorities. There are many complaints from staff and some users about insufficiencies, particularly maintenance budgets and supplies. Smaller municipios may simply not have the funds, while for others it may not be a priority. The best functioning Houses are those in departments like Antioquia which have adopted them as their own program.

Administration

9 The data gathered for the Bello Justice House is based on a census administered during the last trimester of 2010 under the umbrella of this project . This number does not include paid staff alone but also volunteers and interns who were working at the Justice House at the time the census was administered. Volunteers and interns are mainly concentrated in the categories for equity conciliators, and psychology or legal office interns. This number may fluctuate per trimester.

ASSESSMENT AND IMPACT EVALUATION OF COLOMBIA‘S JUSTICE HOUSE PROGRAM 7

There are three levels of program administration: national, local, and via inter-institutional committees.

National administration lies with the Alternative Dispute Resolution Mechanisms (MASC) Division

(formerly the Access to Justice Division) of the Ministry of Justice and Law (former Vice Ministry of

Justice within the Ministry of Interior and Justice). It has always been a challenge for the Ministry and

Division to make this oversight effective. The Division has actively promoted its vision of how things should be when dealing with various Justice Houses. It has wisely avoided taking national agencies to task for their insufficient collaboration, but this has not earned it any points with the Justice Houses either. Justice House administration is not any stronger, often entrusted to a contracted coordinator with little power to force even local personnel to do much and also lacking funds and a good information system. As for the various coordinating committees – national, municipal and Justice House

– they often do not exist, and when they do, rarely meet.

COMPARING SUPPLY AND DEMAND: WHAT WORKS, WHAT DOESN‟T,

AND WHY?

Some problematic areas for all Justice Houses

Unmet needs and unattended cases: The assessment team was asked to review NJHP activities against unmet legal needs of target populations, apparently under the impression that information on these needs had been documented. It has not, and thus the team has equated needs with the services requested of Justice Houses, a not completely satisfactory substitution, but the only one available. As noted, these demands seem largely met, but we did uncover a category of ―unattended cases‖ worthy of further attention. These were mentioned in staff interviews, meaning that we have no idea of their frequency, but we do know that criminal cases and complaints not subject to conciliation (possibly overlapping categories) are important among them. Aside from demands presented by users with no idea what Justice Houses can do, the most frequent explanations for demands going unattended is that staff is not authorized to treat them (because they are not concilable) or that the authorized (usually national) official is not present. While such cases may occur everywhere, they are most likely to frustrate clients in smaller municipios, where the correct alternative agency is not present.

Unexpected services: This odd term refers to the presence of activities within Justice Houses not included in the ―standard model.‖ There are several variations on this theme and causes for their presence. One involves the use of overly-large Justice House facilities for other local programs – the argument of the local authorities being ―we are paying for maintenance so we may as well use the unoccupied space.‖ Arguably these additions do no harm – they simply aggravate those with a narrow vision as to what should be there. A second variation involves special programs at one time or another incorporated in some legal vision of the NJHP, but sometimes contested by the Ministry of Justice. A third involves wholly local inventions, also locally financed, inserted to meet a local demand, or to earn the authorities credit with someone. Unexpected services are only mentioned because they have been a source of friction between individual Justice Houses and the Ministry. Many Justice Houses ignore the

Ministry‘s complaints as they have no force behind them; others comply, but in both cases resentment is generated, probably needlessly.

Some further reservations on the high rates of successful mediation: This has been mentioned already and needs little further explanation. The rates are extremely high and probably merit closer examination. It is possible that the conciliators in the Justice Houses are unusually successful or that their clients are extremely receptive to the practices. However, the latter may not have understood the

ASSESSMENT AND IMPACT EVALUATION OF COLOMBIA‘S JUSTICE HOUSE PROGRAM 8

survey question or it may have been asked too soon to determine whether compliance persisted over time. Additional research might be merited.

Adequacy of services to rural and CSDI populations

This responds to specific questions asked by USAID. Generally the answers are positive with two caveats. Rural populations are underrepresented among Justice House users, but once they obtain access they are more appreciative of the services than are others. CSDI scores were the lowest of the four categories of users surveyed, but they are still acceptable. Since CSDI and small, rural municipios largely entered the program in the last phase (2006-present), part of the explanation may lie there, especially as regards attracting more rural users. There are other questions raised by the contrasts: the possible need for better communication strategies especially for rural populations, the need to explore the potential for itinerant Houses, the role of additional programs (called the ―extended platform‖) in keeping CSDI scores high, especially in focus groups where they do better, and the potential for a less standard model in rural, CSDI regions.

Performance differences among Justice Houses

There are differences, and considerable ones, but we could find no patterns except for those separating

CSDI and non-CSDI Houses – usually, but not always showing higher ratings on various questions for the latter. When we separated Houses by category of municipio, we found nearly as much intra- as inter-category difference and also, surprisingly, differences in the scores on several key items within individual

Justice Houses. Some of this may be due to our questionnaire and how it was administered in different

Houses, but we believe the findings (especially the differences within the same House) merit examination. Quite possibly they are related to the complaints about operational issues raised by Justice

House staff, and if so, they could provide the basis for some targeted assistance.

REFLECTIONS AND RECOMMENDATIONS

Three conflicting findings

Judged by the reactions of Justice House users, the program has been a success. The users are getting the services they want and are, for the most part, pleased with the results.

Judged against the opinions of friendly critics, the program has been less successful because it has not accomplished all the functions they believe it should hold or, conversely, has exceeded its mandate.

Evaluated in light of its sustainability and further expansion, the program does face problems, but their dimensions and possibly their solutions vary depending on the view as to what it should be accomplishing.

The facts are indisputable; their implications are not. That has led us to divide the recommendations into two categories; short-term activities that should be done or at least begun before any program roll-out; and medium- to long-range alternatives for further consideration. Three criteria should be used in the latter exercise: desirability, feasibility (will it work regardless of funding and is funding available?) and coordination (will the other partners accept the plan?). Desirability rarely seems in question, but the other two criteria get short shrift in these discussions and they are equally critical.

Short-term recommendations that do not depend on answers to the bigger questions

ASSESSMENT AND IMPACT EVALUATION OF COLOMBIA‘S JUSTICE HOUSE PROGRAM 9

The following activities are recommended and could be funded by USAID or other donors:

Improving information on Justice House operations: this involves two areas – staffing, activities, and results; and budgets. Each system should be integrated at the Justice House level, and could be centralized as well, although it may be easier to do this by stages.

Defining the role of the Ministry of Justice as a facilitator, not an enforcer, and improving its ability to carry this out.

Reaching an interim agreement on ―unexpected services‖ by lessening the restriction on the use of Justice Houses for other activities, so long as they carry out the core functions.

Exploring the issue of unattended services and finding solutions if it indeed is a widespread problem.

Inventorying and evaluating all services, and especially those apart from dispute resolution, conducted by individual Justice Houses.

Evaluating the Centers of Convivencia as a potentially interesting model for smaller municipios.

The bigger issues to be addressed as a prelude to medium and long-term planning

These involve three series of questions:

The Justice House purpose and ―model:‖ What purposes can Justice Houses logically and practically serve and do they differ according to where they are located? Should there be a standardized model, minimum requirements, or, purely local determination of what each Justice

House does?

Funding: How can the demands of the Justice Houses, in terms of budgets and human resources, be coordinated with the other needs and agendas of the potentially participating entities? How can the needs of class 6 municipios and other funds-strapped localities be met?

Organization: What should be the role and responsibilities of the Ministry of Justice vis-à-vis the program and are there any feasible alternatives for program coordination?

There are two extreme solutions, both of which seem impractical if for different reasons.

Proceed with the standard model, accepting the financial burdens and its implications. Costs and the necessary agreements among the parties make this seem impractical, but this option continues to have its proponents.

Leave the situation as is, doing nothing with its ―voluntary‖ organization or with the Ministry‘s current lack of instruments to enforce any standard vision or model. While this may end up as a default solution, it does raise a series of questions: What the program is and if donor support is to be provided, how and to whom? How to encourage its adoption in new departments and municipios and how to address issues posed by existing, more problematic Houses? Leaving the program or the individual Houses on the current course offers certain attractions, but a rational use of resources is not one of them.

Justice House models: a single set of services and service providers, a core set of each, or an entity whose functions and organizations are defined by local authorities on the basis of local needs

Our recommendations lean toward a core set of functions as a prerequisite for inclusion in the program

(and not incidentally for donor funding). However, beyond that, we have identified no negative impact in adding additional activities, especially if local or departmental authorities can fund them. One might

ASSESSMENT AND IMPACT EVALUATION OF COLOMBIA‘S JUSTICE HOUSE PROGRAM 10

think in terms of a certification program for the former, and a competition for the latter – to determine which additional activities have proven most useful to the target population.

Justice House funding: Local, departmental or national and where do the donors fit in?

Unless and until the NJHP occupies a more central role in the Colombian public policy universe, we do not anticipate it getting much more funding from the national government. With luck there will be funds to strengthen the MASC Division‘s ability to carry out some of the suggestions raised here, but certainly not to be a more effective ―enforcer.‖ Hence most recurrent funding will come from local or departmental governments, and some, but hardly as much, from national entities, including the Ministry of Justice. Donors could continue to have a role in building new Houses, once further expansion begins, but more importantly in financing various kinds of technical assistance to improve the program. For departments that adopt the program as their own, and more should be encouraged to do so, additional funding may be less required than specialized assistance. For smaller municipios, some kind of assistance program may be needed. Hence funding would come from different sources, and sources would vary by type of Justice House. As regards our own budgetary analysis, we would note that it incorporates an assumption of full-time Justice House work for both local and national staff. Waiving this assumption, where it does not seem justified by demand, would cut costs from our estimated high budget projections. As long as staff working part-time in the Justice House and devoting the rest of the time to

―normal‖ duties keep predictable schedules, this need not harm service delivery.

Justice House organization: Who oversees the process and how much control should they have over its evolution?

There are several roles suggested for the Ministry, including enhanced enforcement mechanisms.

Program control could also be devolved to the departmental level, but this adds other complications.

Informal devolution may work in some cases but the NJHP has value as a national program.

CONCLUSIONS

The program has not reached the proverbial critical juncture, but it is at a point where new understandings seem necessary. Since it has worked until now, the most important warning is not to change so much as to undermine its effectiveness. As much as the services it provides could be improved, they are already appreciated by those using them because the Houses afford them services the state has never provided to these groups before.

ASSESSMENT AND IMPACT EVALUATION OF COLOMBIA‘S JUSTICE HOUSE PROGRAM 11

IMPACT ASSESSMENT AND EVALUATION OF

Introduction

PURPOSE

In 2010, USAID/Colombia under Contract No.: AID-514-TO-10-00001 directed Management Sciences for Development, Inc. (MSD) and its subcontractor, Partners/Colombia, to conduct an assessment of

Colombia‘s Justice House Program. The request was based on recommendations from an evaluation of

USAID‘s Justice Reform and Modernization Project (2010) which provided support to the program. The assessment‘s purpose is to guide USAID and the Government of Colombia in their future actions vis-à-vis the Justice Houses. While both the 2010 evaluation and other observers agreed from the start that the program has been useful, the fact that it had never been adequately evaluated,10 that its content and objectives have varied over time, and that there were observable tensions and problems motivated this request.

This report draws on a prior review of available documents and extensive fieldwork on the operations of 21 of the 73 Justice Houses in existence as of early 2011.11 The assessment was to focus on the quality of services provided to urban and rural users of the Justice Houses and, to the extent possible, to compare this with the service needs of the target population. Specific topics to be covered include:

The program‘s legal framework

The program‘s success in fulfilling its strategic objectives, including reduction of barriers to citizen access to justice; enhancing state presence at the local level; reduction of judicial backlog through use of alternative dispute resolution mechanisms; and development of programs to prevent domestic violence

Staff strengths and weaknesses especially as regards assimilation of skills, abilities and knowledge to conduct the program

Adequacy of administrative, logistical and technological support

Evaluation of services provided vis-à-vis the contextual setting of the Justice Houses and especially those located in conflict zones and rural areas.

The assessment was to include a series of recommendations for improving the program based on the findings from the fieldwork in particular and focusing on the specific themes listed above.

METHODOLOGY

The research for this report relies on a multi-dimensional methodology. In addition to a review of prior documentation and evaluations, which served largely as background, the assessment draws on a series of quantitative and qualitative inputs:

10 Both the Ministry and USAID have conducted various partial assessments (see entries in Bibliography), but for various reasons it was decided that a further, more systematic assessment was required. As a result, three separate studies were commissioned more or less at the same time, each with a slightly different focus. Our team was able to talk to the author of one of them, José María Mejía Wills, and some of his findings are incorporated in this report.

11 By early 2012, four more Houses had been added, bringing the total to 77.

ASSESSMENT AND IMPACT EVALUATION OF COLOMBIA‘S JUSTICE HOUSE PROGRAM 12

A series of structured interviews (3,188) with service users in 19 Justice Houses. These interviews, collectively termed the poll, did not involve a random sample, but rather utilized (as explained in the methodological annex) a somewhat less systematic selection of individuals who had visited the Justice Houses in the 19 locations.

A series of structured interviews (2,302) based on a random sample of ―users‖ of Justice Houses in two locations (Villavicencio and Villa de Socorro). The same questionnaire was applied as for the poll, the only difference being the more systematic selection of respondents.

A series of structured interviews with non-users – individuals residing near each Justice House who had not accessed their services. Comprising responses from 586 individuals in the polland

158 from the random samples, these constitute a sort of control group. While it would be hard to say the respondents had not been affected by the Justice House work, they do offer an alternative view of its impacts.

A series of 84 structured interviews with personnel in the 21 Justice Houses, asking them about the conditions of and preparation for their work, as well as their general impressions as to its impact.

A census conducted quarterly by the Ministry of Interior and Justice since 2002.12 This provides an alternative source of data on Justice House activities, composition, and uses.

Thirty-five focus groups held in the 21 Justice houses with a particular emphasis on women and ethnic minorities. Results have been partially quantified but largely appear as qualitative additions to elaborate on the more quantitative inputs.

Discussions with a persons interviewed (Annex 4) to draw in more informed critiques

An analysis of the legal framework for the program and its evolution over time.

Observations by the team conducting the fieldwork.

Combining all these inputs has proved a challenge, especially as there are a few contradictions among the various sources. This is not only true of the juxtaposition of qualitative and quantitative accounts but also as regards the main sources of information on services – the Census, the 19 ―non-random‖ samples within the poll, and the two random samples. Part of the explanation, especially as regards differences between the Census and the survey results, is due to differences in methodologies and how events were categorized, but there may be other explanations. The differences were not dramatic, however, and consistency as regards general findings (related to population characteristics, types of services provided) suggest that overall the poll, if not randomly constructed, can still be considered as a reasonable reflection of the Justice Houses as a whole. The focus groups, and especially those with women, also differed in their findings vis-à-vis the structured interviews, but as discussed in later sections, this is to be expected. Participation in a group discussion tends to modify individuals‘ responses and also allows the opportunity to provide answers and details not possible in a structured questionnaire.

ORGANIZATION OF REPORT

Following a section on the development of the Justice House Program since its initiation in 1994, the topics covered are as follows: a review of findings on the demand side (how target populations use the

Justice Houses and with what results); a section on the supply side (reviewing organization, and other issues affecting the supply of services); a section attempting to compare and contrast supply and demand to…