ATTACHMENT 11 ZdravPlusII Mid-Term Evaluation Report.pdf

Mid-Term Evaluation of USAID/CAR Project

Quality Public Health and Primary Health Care in the

Central Asian Republics

“ZdravPlus II”

Team Members

Catherine “Tina” Cleland, Team Leader Jill Boezwinkle, USAID/GH

Karen Cavanaugh, USAID/GH Forest Duncan, USAID/E&E

James Heiby, MD, USAID/GH

April 17, 2008 ii

Table of Contents

Page

Acknowledgements iv

List of Acronyms v

Executive Summary vii

I. Background and Context 1 II. Problem Statement and Theory of the Intervention and

Design of the Project 2 III. Purpose of the Evaluation 4 IV. Country Findings 4

A. Summary 4 B. Organization of the Project 5 C. Description of Country Specific Analysis 6

1. Summary of Performance Findings

2. Recommended Priorities for Health System

Strengthening over Next Two Years

3. Priorities for 2010 and Beyond

4. Expectations of Movement Along the Development Continuum over the Next Two Years

D. Individual Country and Regional Reports

1. Kazakhstan 7

2. Kyrgyzstan 11

3. Tajikistan 15

4. Turkmenistan 18

5. Uzbekistan 21

6. Regional 25

V. Technical Comments 26 VI. The Value of the Regional Approach 30 VII. Lessons Learned and Best Practices 31 VIII. Project-wide Recommendations: Priorities for Health 34

System Strengthening over next Two Years IX. Conclusion 37 iii

ANNEXES

A. Kazakhstan Country Findings A1 Kazakhstan Movement Along the Development Continuum B. Kyrgyzstan Country Findings B1 Kyrgyzstan Movement Along the Development Continuum C. Tajikistan Country Findings C1 Tajikistan Movement Along the Development Continuum D. Turkmenistan Country Findings D1 Turkmenistan Movement Along the Development Continuum E. Uzbekistan Country Findings E1 Uzbekistan Movement Along the Development Continuum F. Scope of Work for Mid-term Evaluation of USAID/CAR Quality

Public Health and Primary Health Care in Central Asia (ZdravPlus II) Project, November 2007 G. Methodology for Project Evaluation H. Generic Interview Guide I. List of people interviewed by country J. USAID Country-Level Foreign Assistance Extended Framework:

Criteria for country classifications of developing, transition and sustaining partner

K. ZdravPlus II Contract Scope of Work Attachment 6: Background Information and Mobilization Plan (Not included – available at

USAID/CAR)

L. USAID CAR Health Portfolio Results Framework M. Original Concept Paper for ZdravReform in CAR: Lessons Learned and Next Steps in Health Reform for Central Asian Republics, Prepared by ZdravReform staff for USAID/CAR Michael Borowitz, Sheila O’Dougherty, Cheryl Wickham, Grace Hafner, Juliann Simidjiyski, Cari Ann VanDevelde, Mark McEuen, October 1999.

N. Bibliography of Documents Reviewed for the Mid-term Evaluation O. Obligations by Funding Source P. ZdravPlus II PMP Tables (2004-2007) Q. ZdravPlus II Technical Paper: Regional/Technical Overview R. ZdravPlus II Technical Paper: Implementation Strategies S. ZdravPlus II Major Activities by Geography T. ZdravPlus II Country Tables showing progress from Independence through the three phases of USAID funding for Zdrav U. ZdravPlus II Lessons Learned V. Indicator Graphs for Each Country (5 total) W. ZdravPlus II Response to Evaluation Team’s Questions X. Partnership Defined Quality Methodology Y. CORE Initiative Capacity Assessment Tool iv

Acknowledgements

The members of the evaluation team, Tina Cleland, Jill Boezwinkle, Karen Cavanaugh, Forest Duncan, and James Heiby, wish to express our immense and heartfelt gratitude to the USAID/CAR staff in Almaty, Astana, Bishkek, Tashkent, Dushanbe, and Ashgabat;

to the ZdravPlusII staff in Almaty, Astana, Karaganda, Dushanbe, Bishkek, Ferghana, Issyk Kul; to our interpreters Larissa Yurassova (Kyrgyzstan and Kazakhstan), Djalilova Munavara (Tajikistan), Elena Zdravomyslova (Turkmenistan), Gulruh Caron (Uzbekistan); to our drivers Stawitski Slava, Guralevick Igor, Frank Alexey and Glazkov Sergey (Kazakhstan), Kiriyak Vassily (Kyrgyzstan), Asdov Nodir (Tajikistan), Vladimir Litvinov (Turkmenistan), and Tahoev Ibragim (Uzbekistan); and to all of the many patients, physicians, nurses, midwives, feldshers, professors and others throughout the region who so graciously shared with us your time, documents, insights and wisdom. We arrived in the middle of the coldest winter in Central Asian history and you warmed us with your renowned hospitality. You opened the doors to the health system for us and helped us to understand the challenges you have faced and the many accomplishments of which you are rightfully so proud. We thank you for your patience and good humor in the face of our considerable language difficulties, many requests, and inevitable snafus.

We ask for your understanding of any errors we may have made in our interpretations of the information you shared with us. We thank you for the privilege of learning about your countries and seeing the region’s health system through your eyes.

The team also wants to acknowledge ZdravPlus II’s considerable effort to prepare papers requested by the team to facilitate our analysis. These papers supplement the PMP reports, and monthly, semi-annual and annual progress reports submitted by the ZdravPlus II. They were excellent resource documents for the team and should become part of the permanent legacy of the project. For this reason, we have included them as Annexes Q, R, S, T, U and W.

v

List of Acronyms

ADB Asian Development Bank AIDS Acquired Immune Deficiency Syndrome AH Arterial Hypertension BBP Basic Benefit Package BWAK Business Women’s Association of Kazakhstan CAH Swiss Red Cross Community Action for Health (Kyrgyzstan) CAR Central Asian Republics CBO Community Based Organization CDC U.S. Centers for Disease Control CGD USAID Center for Global Development CHSD Center for Health System Development (Kyrgyzstan) CI Cardiology Institute (Kazakhstan) CME Continuing Medical Education COE Center of Excellence (Tajikistan) COM Council of Ministers (Uzbekistan) CORE CORE Initiative Capacity Assessment Tool CPG Clinical Practice Guideline CRH Center for Reproductive Health CTO USAID Cognizant Technical Officer CQI Continuous Quality Improvement DFID United Kingdom Department for International Development DIC Drug Information Center (Kazakhstan) DOTS Directly Observed Therapy – Short Course EBM Evidence Based Medicine EBG Evidence Based [Clinical] Guidelines EML Essential Medicine List EDIN Eurasia Drug Information Network (Uzbekistan) FBO Faith Based Organization FGP Family Group Practice (Kyrgyzstan) FGPA Family Group Practice Association (Kyrgyzstan) FM Family Medicine FMA Family Medicine Association (Tajikistan) FMSA Family Medicine Specialists Association (Kyrgyzstan) FSU Former Soviet Union GAVI HSS Global Alliance for Vaccine and Immunisation Health Systems Strengthening GDP Gross Domestic Product GH USAID Bureau of Global Health GNI Gross National Income GOK Government of Kazakhstan GOKR Government of Kyrgyzstan GOT Government of Turkmenistan GoU Government of Uzbekistan GP General Practitioner (Uzbekistan) HA Hospital Association (Kyrgyzstan) HIS Health Information System HIV Human Immunodeficiency Virus HMN International Health Metrics Network HR Human Resources IDF Institutional Development Framework IIP Integrated Improvement Programs IMCI Integrated Management of Childhood Illnesses IRTT World Bank Health Sector Institutional Reform and Technology Transfer Project (Kazakhstan) ISQUA International Society of Quality Assurance ISRR Institutional Structure, Roles and Relationships JPIB Joint Program Implementation Board (Uzbekistan) KAFP Kazakhstan Association of Family Practitioners MAC Medical Accreditation Committee (Kyrgyzstan) MCC Millennium Challenge Corporation MCH Maternal and Child Health MDR-TB Multi-Drug Resistant Tuberculosis M&E Monitoring and Evaluation MHIF Mandatory Health Insurance Fund (Krygyzstan) MHIG Mahalla Health Initiative Group (Uzbekistan) MIC Medical Information Center MOF Ministry of Finance MOH Ministry of Health MOHMIT Ministry of Health and Medical Industry of Turkmenistan vi

MOEBP Ministry of Economy and Budget Planning (Kazakhstan) NGO Non-governmental Organization OCAT Organizational Capacity Assessment Tool ODB Outpatient Drug Benefit ODBP Outpatient Drug Benefit Program OECD Organisation for Economic Co-Operation and Development OHD Oblast Health Department PAL Practical Approach to Lung Disease (Finnish, Kyrgyzstan) PDQ Partnership Defined Quality PED U.S. Program for Economic Development PGI Post-Graduate Institute for Physicians (Kazakhstan) PHC Primary Health Care PMP Performance Monitoring Plan PP/PM Per Person/Per Month PPS Prospective Payment System QA Quality Assurance QI Quality Improvement QIP Quality Improvement Project (Uzbekistan) RHC Rayon Health Committees RMIC Republican Medical Information Center (Kyr) RRS Rayons of Republican Subordination (Tajikistan) RTI Research Triangle Institute SES Sanitary Epidemiological Service SGBP State Guaranteed Benefit Package (Kyrgyzstan) SHCDP State Health Care Development Program (Kazakhstan) SM Safe Motherhood SOW Scope of Work STLI Scientific Technology and Language Institute STI Sexually Transmitted Infection SVP Rural primary health care centers (Uzbekistan) SWAp Sector- Wide Approaches TA Technical Assistance TB Tuberculosis TIAME Tashkent Institute for Advanced Medical Education (Uzbekistan) TOT Training of Trainers TSMU Tajikistan State Medical University UNICEF United Nations Children’s Fund US United States of America USAID United State Agency for International Development USG United States Government VHC Village Health Committee WB World Bank WB PIU World Bank Project Implementation Unit (Tajikistan) WHO World Health Organization WHOI-EURO World Health Organization International-Europe WTO World Trade Organization vii

Executive Summary

A. Background and Objectives

The Quality Public Health and Primary Health Care in Central Asia (ZdravPlus II) project is the third of a series of projects (“Zdrav”) that began in 1994 with the same prime contractor (Abt Associates, Inc.) to support the reform of the Soviet-era health systems in the Central Asian Republics. The countries of Kazakhstan, Kyrgyzstan, Turkmenistan and Uzbekistan have been part of the project since the beginning. Due to a civil war, assistance to the country of Tajikistan did not begin until 2005. The current contract period is five years (January 7, 2005-December 6, 2009) for a total estimated cost of $30,551,021.

The USAID/Central Asian Republics (CAR) Mission commissioned a mid-term project evaluation to be conducted by a five-person team (“the Team”) starting in January 2008, with four objectives:

• Assess the contractor’s performance.

• Review the regional approach used in this contract.

• Summarize lessons learned.

• Identify priorities for USAID assistance in health systems strengthening during and beyond the contract period.

B. Context

The objective of USAID assistance was to restructure the health care system to have a strong primary care foundation. The strategy was summed up as “inverting the health care pyramid,” whereby most resources were concentrated at the bottom of the health care system (at the primary care level) rather than at the top (at the hospital level). To achieve this, the quality of primary care had to be improved, resources redirected and patients educated about the benefits of self-care and primary care.

Since the beginning of the Zdrav series an overall framework of four components has been used for organizing country specific reform strategies. The components are

1) Stewardship – The government’s role in health—policies, laws, and institutions;

2) Resource use – Financing, organizing, managing and staffing health services;

3) Service delivery - The combination of services, providers and strategies for delivering quality patient care; and

4) Population and Community Health - Education and empowerment of individuals and communities to care about their health and health care.

viii

C. ZdravPlus II Performance Assessment

1. Overview:

In the context of each country’s opportunities and limitations, Zdrav Plus II has made significant progress with its comprehensive, yet tailored, approach to health system restructuring. They are fully meeting and often exceeding the technical requirements of the USAID contract. The success achieved by this relatively small technical assistance project is due to a number of factors. For one, ZdravPlus II has been unusually diligent about coordinating its work with major donors, such as the World Bank and the Asia Development Bank, effectively leveraging very large resources.

Knowledgeable officials from host countries and donors describe ZdravPlus II as a critical collaborator, highly valuing its in-depth knowledge of the local context, superior expertise in health reform issues, experience in change management and details of implementation as well as excellent working relations with governments. Among the enabling factors behind this performance, the Team would identify the long continuity of the Zdrav series, USAID’s flexible management in allowing ZdravPlus II broad latitude to pursue reform, and sustained visionary leadership. These attributes have allowed ZdravPlus II to leverage country health system resources, move expeditiously when opportunities arise and collaborate with a large number of international partners.

ZdravPlus II has recently entered a phase of accelerating value added, with multiple reforms entering large-scale implementation across the region. This expansion suggests that host countries are increasingly open to enacting reforms, based on successes they are seeing in the region, particularly in Kyrgyzstan, and their confidence in ZdravPlus II’s advice. ZdravPlus II assistance may also be growing more effective based on their experience testing reform models.

The Zdrav series of projects is remarkable for its ambitious goals, long-term continuity, and potential for large-scale, institutionalized benefits for health. It has pushed well beyond the boundaries of previous USAID health reform initiatives globally. That such a wide range of promising changes was supported with such a modest investment by USAID is a tribute to the dedication and skills of both the Zdrav and USAID/CAR managers.

2. Performance against Performance Monitoring Plan (PMP) Indicators

Analysis of the PMPs for the five CAR countries shows that with few exceptions ZdravPlus II has met or exceeded targets in all countries. In some cases, the targets for 2009 have already been met. The PMPs were found to not always be useful in gauging the progress of reform in the country.

ix

3. The Four Components

ZdravPlus II’s approach to project implementation is summed up as a top-down/bottom-up strategy. The four components are useful for teasing apart the layers within the system.

a. Stewardship

Achievements in the stewardship component have been an area of consistent strength. From the outset of its work, Zdrav effectively gained trust from host governments along with a willingness to be open to Zdrav’s ideas about how to proceed with reforms. The high level of trust is so pervasive that host country leaders have adopted ZdravPlus II’s ideas as their own, greatly facilitating the advancement of reform through the stages of legal and regulatory change to pilot testing. All governments regard ZdravPlus II as an invaluable advisor.

The changes that have resulted from ZdravPlus II’s work in this component are quite variable and depend upon the governance and recent history of each country. Kyrgyzstan and Kazakhstan have been most active in reforming policies and institutional roles, achieving changes across a wide spectrum of reform issues. In Kyrgyzstan, the legal framework for reform is now firmly established.

Kazakhstan still has significant issues to address.

Uzbekistan was slower to open up to health reform and its progress in stewardship is limited to legislation related to resource use. In Tajikistan, the government has passed a few measures that are consumer-oriented, such as introduction of a basic benefit package and co-payments. Turkmenistan has adopted policy changes related to nine clinical areas, mostly maternal and child health.

b. Resource Use

The acceptance of new financing systems and roles for facility managers has been uneven. Kyrgyzstan, Kazakhstan and Uzbekistan have implemented capitated payment for primary care, pooling of health service delivery funds under one institution at either the national or oblast level, a single payer system and establishment of health care facilities as independent legal entities, giving decision making authority to managers. Of these four indicators of reform progress, only Kyrgyzstan has a fully functional system. Kazakhstan does not yet have information systems in place that enable managers to perform their new functions. Uzbekistan is capitating only rural facilities nationwide, and facility managers have little actual autonomy. In Tajikistan capitated payment has been implemented in eight pilot rayons. In Turkmenistan the project has supported new management systems, in particular health information, health financing and provider payment systems.

x

c. Service Delivery

In this component, the differences in country priorities are pronounced, and demonstrate why responsiveness to host country conditions has been such an important factor in ZdravPlus II’s success. They have been flexible about how reforms should be sequenced, and have concentrated on areas where they could gain the most immediate traction.

Family medicine, EBM and continuous quality improvement (CQI) are at the heart of the reform agenda. The project’s efforts to help change physician practice, however, are proving to be quite challenging. Kyrgyzstan and Kazakhstan are promoting family medicine and evidence-based medicine (EBM) as national priorities, but the medical academic establishment continues to resist EBM. This situation is impeding the progress of reform.

In Kyrgyzstan low physician salaries are creating a serious problem of attracting and retaining providers for rural areas. ZdravPlus II is assisting the Kyrgyz government in developing options for mitigating this crisis.

In Kazakhstan, ZdravPlus II designed integrated improvement projects (IIP) to combine the following at one facility: patient training, CME, CPG implementation and quality monitoring. The Team found this approach to be potentially very influential globally.

Current conditions in Tajikistan are such that the ZdravPlus II program has concentrated on improvements in service delivery. Two centers are combining the use of evidence-based medicine CPGs with practical training in clinics and interaction with patients. They are also capitalizing on the unusually strong leadership at the medical academy to make significant in evidence-based medicine.

In Turkmenistan, host country officials and donors find that ZdravPlus II’s depth of knowledge about program implementation in the Turkmen health system to be a unique strength. ZdravPlus II has used a very small budget effectively to support maternal-child health, introducing innovations in a highly conservative system.

In Uzbekistan, ZdravPlus II is working on health services improvements through collaboration with other donors.

d. Population and Community Health

Kyrgyzstan is the only country where there has been significant progress in establishing national membership organizations for professionals, associations of groups working in the health sector and community-based organizations for involving the public in the health care system and their health care. The shared vulnerability is the lack of plans for ongoing financial sustainability.

xi

The role of NGOs in Kazakhstan is limited, but it has been accepted. ZdravPlus II funded the Kazakhstan Association of Family Practice (KAFP) to promote the new profession of family medicine and evidence-based approaches to disease treatment. Long term financial viability of KAFP is a source of concern.

In Tajikistan, there are modest, but encouraging signs that the government will be receptive to ZdravPlus II’s assistance in educating the public about the reforms and promotion of personal health.

Turkmenistan has one program for health promotion which operates in only part of the country.

Due to the climate in Uzbekistan regarding community action, ZdravPlus II’s work has been scaled back and refocused through different channels.

4. Performance by Country

Kazakhstan: As it moves to modernize its system, the GOK has expressed a sense of urgency about the need for the resources of ZdravPlus II to be available to it, as they are still greatly dependent on the project for analysis and guidance.

ZdravPlus II achievements in Kazakhstan are most significant in the areas of stewardship and improved resource use. The greatest challenges for ZdravPlus II lie in the complexity of transitioning to a new service delivery system and the resistance to an active NGO role in gaining public awareness and enthusiasm for the reforms. They are testing models that integrate all the components of their multi-faceted program, including financing and payment, family medicine, evidence-based medicine, health information systems and outpatient drug benefits. But the new system is not yet fully operational across any oblast.

To protect the USAID investment in the Kazakh health reform program, the project’s continued presence is essential for at least the next two years. The Team suggests that the GOK be approached about assuming responsibility for future funding of activities in Kazakhstan under the direction of USAID.

Kyrgyzstan: ZdravPlusII has made impressive contributions to the health system in the Kyrgyz Republic through all four components in fundamental and significant ways. By the end of this project, the Kyrgyz Republic health system will operate in fundamentally different and better ways because of ZdravPlusII’s engagement. The remaining challenges are to reform the public health system and ensure the sustainability of local institutions that will bear the responsibility of continuing to operate and strengthen the health system. The long-term viability of the health system will depend on continued political support, engagement of strong civil society entities, increased public spending on health, solving the problem of workforce migration, and solutions that engage the private sector and respond to the needs and interests of urban as well as rural residents.

USAID support to address these issues continues to be needed.

xii

Tajikistan: The Tajik government is just beginning to restructure and strengthen primary care. There is little human or institutional capacity in the country to achieve this;

but where pockets of capacity exist, ZdravPlus II is leveraging them to make exceptional gains. ZdravPlus II has established itself as the primary policy and legislative adviser to both the Tajik government and other donors.

In the stewardship area, legislation has been passed on family medicine, physician capitation, a basic benefit package and co-payments. The service delivery component is the heart of the project in Tajikistan. There are two important demonstrations showing how vertical programs can be integrated into a family medicine practice. There are good prospects for continued progress in the service delivery component because several prominent academic leaders are working with ZdravPlus II to modernize clinical practice.

Turkmenistan: From the beginning of the Zdrav series, the project has maintained an effective working relationship with the government, even as many other assistance programs made little progress or were ended. To a large degree, Zdrav’s continued presence in Turkmenistan since 1994 reflects its responsiveness to the government and appreciation for the benefits they received from learning about the experiences of other ZdravPlus II countries, and the adaptation of tools from these countries.Zdrav’s work on IMCI and SM provided the clinical subject matter for successfully introducing EBM, eliciting a request to expand the EBM training program. Senior officials expressed a high degree of trust in the project staff. Recent political changes seem likely to accelerate the reform process.

Uzbekistan: With the assistance of ZdravPlus II and other donors, Uzbekistan has almost completed the roll-out of a rural primary health care program. Important reforms have been achieved in resource use and service delivery. These initiatives, coupled with clinical training and introduction of quality improvement systems, are expected to significantly improve services and efficiencies. The changes have led to increased capacity to make budget decisions at the oblast level and have also given facilities more (though still quite limited) control of their finances and management. An urban version of this approach is currently being piloted.

C. The Regional Approach

The Team found that the regional approach taken under the Zdrav initiative had many benefits –technical, economic and strategic. However, under ZdravPlus II funding of the regional activities has been greatly reduced. As health systems reform moves from policy debates to the details of implementing complex changes, the Team finds the technical relevance of work across countries to be of growing, not diminishing, importance.

D. Lessons Learned

The team defined a lesson learned as a feature of the project that has been particularly effective in the achievement of USAID’s objectives, both in the health sector and beyond.

• Reform takes time.

xiii

• Well managed projects attract capital to the health sector from the host country and other donors.

• The strongly supportive relationship between ZdravPlus II and host country counterparts is a proxy for US relations, creating an immense amount of goodwill for the United States and USAID in the region.

• Models of good governance, Treasury system reform and civil society in the health sector are transferable to other sectors within the government.

• Donor harmonization, project coordination and integration will maximize the impact of each donor’s efforts and reduce the host country costs of servicing the donors.

• Host countries and the World Bank view the continuity of USAID’s role in successful reform efforts as a form of insurance for their investments due to the technical expertise of ZdravPlus II.

• Self-contained, vertical disease-specific programs, such as those for HIV/AIDS and TB, are not making use of all the available resources for treatment. ZdravPlus II’s success at grounding these programs in the primary care system has given providers access to expanded resources, improving patient care and outcomes.

• A bottom up and top down approach brings together the community and service delivery levels and the political and legal levels of reform to creating lasting change and ownership of reform efforts.

• Key stakeholders are the engine of change. Well-placed advocates accelerate change once institutions are in place.

• Creating regular forums (i.e., seminars, conferences, workshops) for sharing ideas and solving problems across countries and within countries facilitates understanding of the reforms and creates linkages.

• Complex reform initiatives can be tailored to the abilities of each country.

• Maintaining the role of a technical resource, not aligned with a political agenda, enables the contractor to be flexible and agile as governments undergo change.

• Engagement of civil society organizations and professional associations contributes to country ownership and sustainability.

E. Priorities for USAID Assistance in Health Systems Strengthening During and Beyond the Contract Period

This project has helped Central Asian countries make tremendous advances in structuring their health systems to operate efficiently and to respond to the health care needs of their population. Selected strategic recommendations relating to areas that need a boost are listed below. The report includes additional individual country recommendations for the next two years and beyond which are not included here.

1. The Last Two Years of the Contract Period (Unless a specific country is identified, the recommendation applies to all countries.)

• Determine the prospects for success in reforming the SES system before proceeding further. (Kazakhstan, Kyrgyzstan)

• New strategies are needed to improve the prospects for adoption of EBM FM.

xiv

• Address areas of vulnerability before withdrawing USAID support: 1) measures to sustain the non-governmental organizations that have been spawned by this project, and 2) ways to link retention incentives with human capacity development. (Kyrgyzstan)

• Develop a fully integrated model of health reform in at least one entire oblast in Kazakhstan. Expand this model to at least one other entire oblast (Kazakhstan)

• Begin developing a PED-type arrangement with the GOK so that it can have on-going access to ZdravPlus II technical assistance and USAID can maintain high visibility in the improvement of health care in Kazakhstan.

• The service delivery component should continue to be given highest priority in project activities. (Tajikistan, Turkmenistan, Uzbekistan)

• Expand the number of CPGs to be piloted at primary care centers. (All)

• Incorporate the “improvement collaborative approach” into quality improvement

• Use assessment based approaches to strengthen health promotion activities.

• Begin using the Health Metrics Network assessment tool and guidelines.

• Build capacity to manage complex issues that financing reforms will bring,

• Organize management improvement collaborative among polyclinic managers.

• Apply CQI and EBM to TB and HIV/AIDS in new pilots.

• Establish tutorials for health managers based on financing and service delivery implementation experiences.

• Establish a regional fund for conferences and information exchanges among peer stakeholders in the region.

• Invest in analysis of project experience (successes and failures) and globally disseminate lessons learned.

• Continue to support ZdravPlus II’s role as a collaborator and technical adviser to other donors.

• Align project structure with consensus framework on health system components.

2. 2010 and Beyond

• Expand stewardship and population/community health initiatives in Tajikistan, Turkmenistan and Uzbekistan.

• Establish support and capacity for an institutional structure for pooling of funds for health services delivery and financial management autonomy for health facility managers. (Tajikistan)

• Expand clinical practice improvements using EBM CPGs. (All countries) xv

I. Background and Context

When the Soviet Union broke up in 1991, each country in the USSR gained independence and assumed responsibility for its economic and political future. However, health reform had actually begun earlier in the USSR; when in 1987, management of the health care system was decentralized to the republic level. While health care management had devolved from Moscow, the structure of the Soviet system was entrenched as was the notion that the state was responsible for the social safety net, giving citizens entitlement to health care services “free of charge.”

The Soviet system was based on the Semashko model of health care, a centralized system where the state owned and operated the health care facilities and all health care personnel were employees of the state. The concept of the Semashko system was that primary care was the foundation for a strong health care system. But over the years, the system had become greatly distorted and financing had concentrated at the hospital level with more than 70% of funding going to the highest level of care. Also, the supply of physicians per population was above international norms; treatment plans were heavily medicalized; and there was a predominant reliance on specialist care. Further weakening the system, the running cost of the health sector had always been under-funded since it was viewed as a “consuming” sector, rather than a “producing” sector. In order to get service and medications, patients often had to offer gratuities or “under-the-table” payments to providers. The combined result of under-funding along with distortions favoring specialty and hospital care, was that primary care was starved of both funds and talent. Reflecting the lack of capacity for early diagnosis and treatment and failure of the population to take responsibility for its own health, the health status of the population was in steady decline.

Despite the fact that the sector was under-funded, the state was wasting its scarce resources and the population was absorbing the brunt of the system failures.

In 1994, USAID funded ZdravReform, the first in a series of three projects to help the Central Asian Republics to reform their health care systems. The first project offered assistance to Kyrgyzstan, Kazakhstan, Turkmenistan and Uzbekistan. Now in its third contract period, the Quality Public Health and Primary Health Care in Central Asia (ZdravPlus II) project is funded through 2009 and includes Tajikistan in the scope of work. The current contract period is five years (January 7, 2005-December 6, 2009) for a total estimated cost of $30,551,021. In this report, the group of projects implemented by the prime contractor, Abt Associates, will be referred to generically as “Zdrav.”

The objective of the assistance has been the same since 1994 and is summed up nicely in a project concept paper as follows:

“The restructuring of the primary care delivery system is considered to be a central component of any health reform effort aimed at improving population health status through a strengthened system of primary care. Restructuring the primary care delivery system also provides the conditions necessary for other aspects of health reform, such as the introduction of many modern clinical protocols, the implementation of new provider payment methods, and increasing population involvement and choice.” [Conceptual Foundations for Central Asian Republics Health Reform Model, September 1999]

II. Problem Statement and Theory of the Intervention and Design of Project

In preparation for this evaluation, the team requested ZdravPlus II to submit background information on selected topics. To establish the development theory for Zdrav’s health system reform intervention, the team asked this question: “Provide a statement of the problem when Zdrav was first funded. What was it that USAID set out to fix or change in CAR?” The response follows:

“ZdravReform was first funded in 1994. Early in ZdravReform, three major problems were identified:

a. The very large, overly specialized, and fragmented health system with enormous amounts of excess capacity was no longer sustainable given the general economic and health budget collapse that occurred after the fall of the Former Soviet Union (FSU). Specifically, the hospital sector was very overdeveloped and fragmented and more cost-effective primary health care (PHC) was very weak or even non-existent.

b. In addition to the problems caused by health delivery system structure and financing, PHC was weak or non-existent due to the lack of family or general practice (all doctors were specialists) and the low capacity of PHC practitioners (catchment area physicians largely serving as dispatchers referring the high majority of their patients).

c. The system was not responsive to the needs of the population and the population was not involved in their health.

It was decided the solution to these problems was the overarching project strategy of inverting the health delivery system pyramid. The early specific interventions were forming a new PHC sector; introducing health financing reform including new provider payment systems; introducing family or general practice and upgrading the skills of PHC practitioners; and involving the population through free choice of PHC practice and health promotion. The scope of the health reforms expanded significantly over time and specific interventions evolved with them, however, they generally continued to be built on this basic foundation.

Towards the end of ZdravPlus I, another underlying or core problem was added:

• While producing results, significant efforts to retrain PHC practitioners and implement PHC-level service delivery improvements (largely through WHO programs such as IMCI) were not creating a critical mass for change in the overall nature of clinical practice.

• The problem was defined as medical leadership and specialists not accepting the changes in clinical practice or allowing PHC practitioners to use their new knowledge or to expand the scope of services in PHC.

The solution decided upon was to work to change the overall nature of clinical practice by broadly promoting evidence-based medicine (EBM) to medical leadership and the development and implementation of new clinical practice guidelines (CPGs) in addition to continuing to introduce family or general practice and specific service delivery improvements in priority program areas.

The overall intended results were to use implementation strategies to introduce conceptual/technical interventions solving or improving the major problems identified in the health sector.

[In the] Performance Monitoring Plan (PMP) tables (Annex P), ZdravPlus II Project Regional/Technical Overview paper (Annex Q), Implementation Strategies paper (Annex R), and matrix tables (Annex S) there is more detail; but in general, intended results were/are as follows:

A. Form new PHC sector, restructure and rationalize hospital sector and shift savings to PHC, and introduce new health financing system increasing both equity and efficiency in individual health services

B. Service delivery:

• Introduce family or general practice to increase capacity and improve service delivery in PHC and gradually expand scope of services in PHC by integrating priority programs

• Specific service delivery improvements in priority program areas by implementing new CPGs or standards or implementing WHO programs using facility level quality improvement techniques

• Broadly promote EBM, develop new CPGs, and rational drug use

C. Greater population and community involvement in their health

D. Improve overall stewardship in the health sector

E. Expand the scope of the health reforms to include next generation areas of improving financing system and service delivery for public health, infectious diseases, and undergraduate medical education.

The indicators that the reforms are sustainable and that countries are self-directing their health care systems are as follows:

A. The intended results outlined above

B. Institutional structure, roles, and relationships appropriately separating functions and allowing the right institution to do the right thing, a well established health purchaser and provider autonomy, delegating functions to NGOs and CBOs, clear roles and relationships at national and regional levels, and capacity in the MOH and other entities for health sector stewardship.

C. [As shown in the] health sector pendulum [included in the Implementation Strategies paper (Annex R)], the health system begins to show its capability to continuously refine and self-adjust the system to adapt to changes in the environment and increase the responsiveness of the system.”

III. Purpose of Evaluation

The project to be evaluated is the Quality Public Health and Primary Health Care in Central Asia (ZdravPlus II) Contract #176-C-00-05-00002-00 implemented by Abt Associates, Inc. and various sub-grantees. This contract is currently in its third year of implementation. The project will end on December 6, 2009.

The purpose of this evaluation is to: (1) assess the contractor’s performance; (2) review the regional approach used in this contract; (3) summarize lessons learned; and (4) identify priorities for USAID assistance in health systems strengthening during and beyond the contract period.

IV. Country and Regional Findings

A. Summary: The team found that since 2005 ZdravPlus II has achieved significant successes in all five Central Asian Republics. They have vigorously continued pursuing the Zdrav strategy launched in 1994 to be a partner to countries as they restructure and strengthen their health care system by building a strong foundation of primary care. Since quality primary care is the end result of a well functioning system which includes a multitude of sub-systems, the breadth and depth of program has been audacious and ambitious. The sub-systems involved are financing, organization, management, health information, clinical training, quality assurance and improvement, monitoring and evaluation and health promotion. The regional system of management with country level teams has been effective in utilizing the intra-regional experience of neighboring countries as models for change while at the same time tailoring the project’s concept and strategy to each country’s priorities and stage of development. They have achieved significant economies of scale and the team found that USAID has earned a high return on its relatively modest investment in health reform in five countries.

B. Organization of the Project. Under the ZdravPlus II contract, the scope of work includes country strategies for the five Central Asian Republics and a three part overall regional strategy that is a continuation of the strategy pursued since the original contract award in 1994. Work plans have incorporated 4 broad areas of activity: 1) the regional strategy, 2) country specific reform strategies, 3) cross cutting issues, and 4) donor/project collaboration and coordination.

The overarching regional strategy is to 1) Continue to Strengthen the Core Health System Functions, 2) Deepen and Expand the Integrated PHC Model, and 3) Expand Strengthening Core Health System Functions and Corresponding Integrated Improvement Programs (IIP) to New Areas. IIPs are sites where linkages and synergies have been developed between core health system functions and improvements in clinical care at the point of service or facility level.

Country specific strategies are organized around four components of health system reform: stewardship, resource use, services delivery, and population/community health.

The regional management structure supports each country program through cross-fertilization of ideas, methods, technical expertise and models of reforms. All project activities are identified under one or more of these components. Definitions of the components are as follows:

• Stewardship: (1) Policy dialogue mechanisms and processes, health policy content, and the legal and regulatory framework; (2) Solidify institutional structure, roles and relationships with the health sector and across sectors where necessary, and (3) Contribute to monitoring and evaluation systems for health reform and health service delivery, (4) Policy marketing and public relations and (5) Donor/project collaboration and coordination.

• Resource Use: (1) Health delivery system structure and human resources planning; (2) Health financing – collection, pooling, purchasing/provider payment; (3) Health information systems; and (4) Health management. This component is very broad. It collapses three major health system functions--health finance, human resources, and health information systems. It also encompasses some elements of the service delivery and governance functions from the health system framework which WHO and USAID now use.

• Service Delivery: This component addresses the provision of quality patient care. There are five main activity areas 1) Medical education and human resource capacity development in family medicine (FM) and maternal and child health (MCH), 2) Evidence based medicine and clinical practice guidelines, 3) Peer review through continuous quality improvement, 4) Health purchaser quality assurance systems, licensing and accreditation,

5) Upgrading buildings and equipment and 6) Pharmaceuticals. In Kyrgyzstan, this component also includes SES reform.

• Population and Community Health: This component is devoted to involving the population in caring for their own health and caring about their health care. The strategy includes two areas of activity 1) Educate the population/communities about health reform and their rights, and 2) Educate and empower the population/communities to be responsible for their health.

During this contract period, the contractor was asked to report by country on progress related to the four components of health systems reform, as well as the Integrated Improvement Programs (IIPs), and the new areas of interest under this contract – public health or SES reform, infectious diseases and medical education.

ZdravPlus II’s country specific work plans are organized around the 4 components of a health care system described above. The work plan generally reads as a statement of the problem and its relevance to achieving the goals of the health reform program, along with a statement of intent to engage in activities that will address the problem. It is not specific as to the steps or timeline for how this will be accomplished. The semi-annual reports illuminate the strategy that was followed or is being followed in accomplishing the objective described in the work plan. In this way, Zdrav has used the work plan as a snapshot of the challenges to be tackled, but has not tied its hands by saying how it will proceed. The semi-annual reports include detailed descriptions of the steps taken to move the reforms along as well as the status of the effort, creating a valuable chronology of the sequencing and step-by-step process that are important for understanding the success behind ZdravPlusII’s strategy.

C. Description of Country Specific Analysis This report focuses on the strengths and weaknesses of the ZdravPlus II project. First, there is a review of project performance in each country, looking at the status of each of the four project components as described above. A summary of performance findings is included in the main body of the report and the full analysis for each country is found in Annexes A-E. The primary focus of the country findings responds to the Mission’s special interest in three areas:

1) The recommended priorities for health system strengthening over the remaining project period through December 2009, and

2) The recommended priorities for country programs beginning in 2010 when the ZdravPlus II project ends,

3) The potential for countries to move along the development continuum from “Developing” to “Transition” to “ Sustaining Partner” over the next two years.

(An analysis for each country is included in Annexes A1-E1).

The first two issues are self-explanatory, but the last question regarding movement along the development continuum needs some explanation. In health, the US Foreign Assistance framework for developing countries calls for helping countries to strengthen health systems and health service delivery. It focuses on encouraging good health governance, including policies that strengthen the state’s capacity to establish appropriate roles for the public and private sectors. Transforming countries have basic services and insurance mechanisms in place and functioning. They may face specific weaknesses in financing, accountability, targeting or legislation. They may need help sustaining and accelerating gains in health status through the complementary efforts of the public and private sectors, NGOs, and civil society. Progress from the developing country category to the transforming country category requires expanding and deepening democracy, strengthening public and private institutions, and supporting policies that promote economic growth and poverty reduction. A full analysis of each country’s position on the continuum of development is found in Annexes A1-E1 following the analysis of country performance in Annexes A-E.

D. 1. Kazakhstan Country Findings

D.1.1 Summary of Performance Findings: The project has had important impact in improving the health sector as well as in improving national governance. Many of the obstacles that arose in project implementation were linked to the operations of systems that still concentrate too much control in the central government and are not sufficiently flexible to support decentralized and autonomous operations at the Oblast level. Zdrav has used its pilots to demonstrate how the national systems are impeding progress in achieving the government’s objectives. At the highest levels of government, their analytical and communication skills are viewed as critical to the success of the reforms, both in achievements to date and for the future.

There has been steady movement in establishing a legal and regulatory basis for health reform and testing models that integrate all the components of the multi-faceted program.

National roll-out is beginning. Given the commitment of government to increase the health budget significantly, the two greatest impediments to rapid modernization of the system are 1) the reluctance of the medical profession to embrace evidence-based medicine, and 2) the lack of technical and management expertise to implement reforms nationwide (including the provisions of the World Bank loan). The government is still greatly dependent on ZdravPlus II for analysis and guidance. To protect the USAID investment in the Kazakh health reform program, the project’s continued presence is essential for at least the next two years.

A complete discussion of Kazakhstan’s performance is found in Annex A.

D.1.2. Kazakhstan: Recommended Priorities for Health System Strengthening over Next Two Years

The Team recommends the following health systems strengthening priorities for the next two years. We base these on key informant interviews with senior MOH officials, other GOK counterparts in other agencies at the national and oblast levels, and medical professionals in the facilities we visited. We present recommendations in the context of continuing, starting, or stopping activities depending upon where FY 08-09 funding remains the same, increases, or decreases.

Assuming USAID maintains the current funding…