Attachment_J.5_Zambia_HMIS_Assessment_Report_2005.pdf

Assessment of the Zambian

Health Management Information System

Project SSG/9 ACP ZA 9/5

Arthur Heywood Erik Nielsen

Stanislaw Orzeszyna

September 2005

ATTACHMENT J.5 ZAMBIA HMIS ASSESSMENT REPORT_2005

c o n s u l t a n t s

P

EURO HEALTH GROU

Tinghøjvej 77 DK 2860 Søborg Denmark Phone: (+45) 3969 6888 Fax: (+45) 3969 5888 email: eurohealth@ehg.dk internet: www.ehg.dk

Content Content Abbreviations Executive summary Background Background

Poverty reduction strategy National Health Strategic Plan 2006-2011 Health Sector Reform Health Management Information System

Objectives of the Assignment Methodology Activities Findings Millennium development Goals

National MDG progress report Health system Indicators Population Data

Vital Registration Data sources

Surveys Epidemiological surveillance Routine HMIS

Human Resources Information and Communication Technology (ICT) HMIS Software Administrative and Management Systems SWOT Analysis

Strengths Weaknesses Opportunities Threats

Key strategic issues

1. Capacity development

2. Implementation of DART principles

3. Information and communication Technology strengthening

4. Effective use of information

5. HMIS staff retention

The Way Forward

1. Revised HMIS

2. Capacity development

3. Improved ICT

4. Increased use of information

Annex 1 Terms of Reference Annex 1 Terms of Reference Comments on the Terms of Reference and Key Issues Annex 2 Persons met Annex 3 Timetable Annex 4 Questionnaires Annex 5 Database Annex 6 - Millennium Development Goals Annex 7 – Review of the implementation of the 1996 HMIS Annex 8 HMIS Forms Annex 9 Presentations of results

Abbreviations

ACP African, Caribbean, Pacific Group of States ADB African Development Bank AFP Acute Flaccid Paralysis AIDS Acquired Immuno Deficiency Syndrome ARI Acute Respiratory Infection ART Anti-Retroviral Treatment ARVs Anti-Retroviral Drugs AU African Union AWP Annual Work Plan CARITAS Catholic Health Initiatives CBHMIS Community Based Health Management Information

System CBHG Community Based Health Groups CBoH Central Board of Health CBHIS Community Based Health Information System CHAZ Christian Health Association of Zambia CHW Community Health Worker CMAZ Churches Medical Association of Zambia CP Cooperating Partners CRS Catholic Relief Service CSO Central Statistical Office CSP Country Support Paper DANIDA Danish International Development Agency DART Decentralised, Action-oriented, Responsive and

Transparent DFID Department for International Development DHB District Health Board DHIO District Health Information Officer DHMT District Health Management Team DHO District Health Office DHS Demographic and Health Survey DPT Diphtheria, Pertussis, Tetanus EC European Commission ECHO European Commission Humanitarian Aid Office EDF European Development Fund EPI Expanded Program on Immunization EU European Union FAMS Financial & Administrative Management Systems GIS Geographic Information System GRZ Government of the Republic of Zambia HAST Integrated HIV, AIDS, STI and TB Program HBC Home Based Care HC Health Centre HIS Health Information System HIU Health Information Unit HIV Human Immunodeficiency Virus HMB Hospital Management Board HMIS Health Management Information System HRD Human Resources Development HSR Health Systems Research HSSP Health Sector Support Program IDA International Development Agency IFMIS Integrated Financial Management Information System

IP In-Patient ICT Information and Communication Technology KISS Keep it Simple and Sustainable LAN Local Area Network LDC Least Developed Countries LIMS Logistic Information Management System M&E Monitoring and Evaluation MCH Maternal and Child Health MDG Millennium Development Goals MoFED Ministry of Finance and Economic Development MoH Ministry of Health MoU Memorandum of Understanding MTR Mid Term Review NDP National Development Plan NGO Non-Governmental Organisation OPD Out-Patient Department ORT Oral Rehydration Therapy PEMFAR Public Expenditure Management And Financial

Accountability Review PEPFAR Presidents Emergency Plan for AIDS Relief PLWHA People Living With HIV/AIDS PPAZ Planned Parenthood Association of Zambia PRSP Poverty Reduction Strategy Paper STI Sexually Transmitted Infection SWAP Sector Wide Approach to Planning TA Technical Assistance TB Tuberculosis TBA Traditional Birth Attendant UCI Universal Childhood Immunization UN United Nations UNCTAD United Nations Conference on Trade and Development UNDP United Nations Development Programme UNICEF United Nations Children’s Fund WHO World Health Organization ZMK Zambian Kwacha ZNBTS Zambia National Blood Transfusion Service

Executive summary

Under the Poverty Reduction Strategy Support (PRBS), the European Union provides sub-stantial funds to Zambia for implementation of poverty reduction policies. The PRSP em-phasises health and has a number of indicators to monitor the health sector in line with the National Health Strategic Plan 2006-2011 strategies and objectives. Strengthening of the Na-tional Health Management Information System (HMIS) will lead to improved poverty reduction monitoring.

This document contains the results of an assessment of the Health Management Information System (HMIS) conducted by the Euro Health Group (EHG) for the European Union in July and August 2005. The second part of the assignment was to formulate a detailed, quantified plan of action to strengthen the HMIS in the country. This plan of action is contained in a se-parate document.

The HMIS assessment was carried out in close collaboration with the Ministry of Health (MoH) and the Central Board of Health (CBoH) during all phases of preparation, field work, data analysis and report preparation. Preliminary findings of the consultation were presented and discussed at two meetings (13 July and 3 August 2005) of the Monitoring and Evaluation Subcommittee of MoH and at two meetings (29 July and 5 August 2005) of the Implemen-tation Review Subcommittee of MoH. The methodology and the results of the mission were also discussed with developmental partners.

To assess the HMIS, a framework of the WHO Health Metrics Network was adapted. Parti-cular emphasis was made on the Millennium Development Goals indicators. Other major re-ference points included the Poverty Reduction Strategy Paper, draft National Health Strategic Plan 2006-2011 and National Development Plan.

The assessment found that the Zambian HMIS is basically functional at all levels of the health system and compares favorably with most African HMISs. There is a defined indicator set and regularly collected data elements. Data collection and reporting tools are available at all facilities and district offices and the flow of information are clearly set out using the “one channel” principle. There is regular quarterly analysis of routine data with reasonably good coverage for service-based indicators of underweight children, measles immunization, supervised deliveries, malaria and TB incidence and TB DOTS coverage.

However, many MDG indicators are not monitored, there is poor integration of vertical programs and administrative information into the routine HMIS, in general quality of data is not checked regularly and coherently and the system of vital registration is weak. Most staff are inadequately trained in HMIS procedures and there is not much faith in the results coming out of the HMIS.

Many of the other subsystems need attention and it is proposed that specific plans are made for HMIS for Hospitals, Finances, Human Resources and community based activities. In addition, the MoH should work closely with the CSO to ensure that the results from the census and household surveys are widely available and that vital registration is strengthened.

Five key strategic issues have been identified to strengthen the HMIS:

1. Capacity development of all cadres of staff, including intensive skills development through in-service and pre-service training programs, upgrading of manuals and study of best practice sites.

2. Return to the 1996 HMIS principles of Decentralization, Action oriented, Responsive and Transparent health information system, and introduction of the information pyramid.

3. Information and communication technology strengthening, through making the database more flexible and strengthening of decentralised information centres that are linked by internet to a central data warehouse.

4. Effective use of information through integration of vertical systems, with improved central coordination between stakeholders and sectors so that the information from HMIS can be used to assess output-oriented performance. Improved action research capacity is needed to improve feedback and dissemination and reduce overlap and duplication.

5. HMIS staff retention, particularly District Information Officers, is needed by improving skills and status and ensuring sustainability of systems, procedures and staff.

Major ideas for the Way Forward include:

• A review of the present system, with adoption of a few “key performance indicators” that ensure that MDG and NHSP indicators are regularly analysed by all levels and local self-assessment is routinely done to satisfy the needs of health providers, vertical programs and service managers

• An intensive capacity development program targeted at all users as well as managers, planners and policy makers

• Strengthening of the ICT, focusing on a revised database and data warehouse, with improved equipment and capacity at provincial and district level information centres.

• Improved use of information for local action that responds to local needs as well as the needs of program monitoring and management improvement.

Promote local action research skills to increase local in-depth analysis of existing records and increased use of Sentinel sites to get quality routine data and up- to- date analysis at local level and improved dissemination and feedback of surveys

• A MOH project will provide Zambian leadership to the HMIS through use of international standards and regular monitoring and evaluation

Background

Poverty reduction strategy

In July 2002, the Zambian Government officially launched its first Poverty Reduction Stra-tegy Paper (PRSP) for the period 2002-2004. The main goal is poverty reduction through sustained economic growth and employment creation. In the health sector, in line with existing National Health Strategic Plan, the PRSP has the following programmatic priorities1:

• Provision of the basic health care package, to reduce morbidity and mortality and contribute to poverty reduction. An important component is cost sharing through fee paying; user fees should not, however, constitute a barrier to the poor accessing public health services;

• New approaches to allocation of financial and human resources to districts to give more weight to poverty issues; community participation and accountability of resources will be promoted; the Sector Wide Approach will be retained;

• Restructuring of the procurement system, to ensure that purchasing of drugs is done more efficiently and on a need basis;

• Improving access to health care in hard to reach and under-served areas as well as for vulnerable groups;

• The following areas are public health priorities:

o Malaria - within the framework of the Roll Back Malaria initiative o HIV/AIDS, TB, and STI (HAST) - through the National HIV/AIDS Strategic Framework, o Integrated Reproductive Health - which includes family planning, safe motherhood, o Adolescent health, abortion and post-abortion care, infertility, and (sexual) violence against women, o Child health - which will aim at reducing morbidity and mortality due to childhood diseases, o Epidemics - improved public health surveillance and control of epidemics, o Hygiene, sanitation, and safe water

• The areas of focus of support services are following:

o Development of physical infrastructure and provision of medical equipment o Human resource development o Strengthening of existing management systems

• The overall goal of interventions in nutrition is to achieve sustainable food and nutrition security among the poor and to eliminate all forms of malnutrition in order to have a well-nourished and healthy population that can contribute to national economic development. The strategies are:

1 Zambia Poverty Reduction Strategy Paper 2002-204, Ministry of Finance and National Planning, Lusaka, March 2002, http://www.cboh.gov.zm/documents/PRSP%20Final%20Document.pdf http://www.cboh.gov.zm/documents/PRSP Final Document.pdf o To incorporate nutrition objectives into development policies and programmes; improve human resources and institutional finance; and establish strong nutritional networks, o To strengthen nutrition care practices for the poor, the HIV/AIDS infected, and vulnerable groups, o To prevent and control specific macro and micronutrient deficiencies and promote appropriate diets and lifestyles throughout the life cycle.

PRSP Cross-Cutting Issues

HIV/AIDS - The prevalence and incidence of HIV/AIDS has reached alarming levels in Zambia. The human toll of AIDS is a tragic reality being experienced by families, communities, and the nation at large. There is no aspect of life that has not directly or indirectly been negatively influenced by the AIDS epidemic. AIDS has become the major cause of illness and death among young and middle aged adults, depriving households and society of a critical human resource base and thereby reversing the social and economic gains made since independence.

Interventions for HIV/AIDS under PRSP

• Reduce New HIV/AIDS Infections:

• Reduce Socio-Economic Impact of HIV/AIDS:

• Expansion of Access to Quality VCT Plus Services:

• Community Home Based Care:

• Anti-Retroviral Viral Treatment (ART):

• Improve the Quality of Life of Orphans and Vulnerable Children (OVC):

• Improving STI Management and Treatment in Zambia to Reduce STI infection:

• Expansion of Access to Quality Prevention of Mother to Child Transmission:

• Prophylaxis against TB:

• Drugs for Opportunistic Infections:

Evaluation - A number of clinical, epidemiological, behavioural, and impact studies related to HIV/AIDS will be carried out. A sentinel surveillance system for HIV and population-based studies will continue to be used to monitor the trend of the HIV epidemic. A system of collecting information from health facilities that is already in place to capture cases of AIDS, TB, and other STIs will be used. Lastly, data from various programmes and ministries will be collated and analysed at the national level.

Gender - Issues of gender play a very important role in developing a sustainable strategy for poverty reduction. In addition to economic factors, the socially and culturally ascribed gender roles have contributed to high poverty levels among women. Women are usually not part of decisions made on resource allocation at household and other levels.

PRSP Evaluation Strategy

Poverty monitoring involves tracking key indicators over time and space with a view to seeing what changes have taken place to the indicators following the implementation of the

PRSP. The central objective of the PRSP is to reduce poverty and evaluations will enable the assessment of the impact on poverty of PRSP interventions.

There are three main aspects of the PRSP evaluation strategy:

• evaluation of the implementation process: to attempt to establish whether programme implementation is done according to design and whether things are working or not;

• evaluation of outcomes to establish what the results are in relation to the original or revised programme goals and

• whether particular outcomes are the result of the interventions under the PRSP or a component of it.

PRSP monitoring indicators for the health sector

Objectives Objectively verifiable indicators (unit)

Overall: To improve the health status of all people in Zambia, especially the poor

Life expectancy %

Infant mortality rate

Maternal mortality rate

Under-5 mortality rate Reduce by 8%

To increase the life expectancy of the population

Infants aged 12-23 months fully immunised by 12 months

To encourage lifestyles that support health

Sexually active adults practising safe sex

Children under-5 with low weight for age

% To create environments that support health

Children who are stunted %

Rural households within 5 km of a health facility

Health workers per 1,000 population

To achieve equity in access to health opportunities

Cots and beds allocated per 1,000 population

Health facilities with 80% of established posts filled

Health facilities with essential drugs always available

Health facilities rehabilitated

To provide assured quality health services

New health posts constructed (according to health sector plan)

Policies developed To provide quality policy and technical guidance to service providers

Policies reviewed

National Health Strategic Plan 2006-2011

A 5-year National Health Strategic Plan (NHSP), for the period 2001-2006 is coming to an end. The preparation of NHSP 2006-2011 takes into account the PRSP/NDP process, the MDGs and new challenges to the basic health care package such as treatment for People Living with HIV/AIDS (PLWHs).

Summary of Key Performance Indicators Measure Indicator Infant Mortality Rate 92 per 1,000 live births Maternal Mortality Ratio 729 per 100,000 live births HIV/AIDS Prevalence 16% (16-49 years) Malaria Incidence 396 per 1,000 TB Incidence 545 per 100,000 population Life Expectancy at Birth (Yrs) 50 (CSO) Health Annual Budget Proportion 10% (average) Health Expenditure per Capita 10.80 USD Per Capita Income 280 USD

The NHSP directly addresses the Health Management Information System. It states that current weaknesses of HMIS are, among others:

• Lack of evidence based decisions

• Undue influence of sectional interests

• Weak demand for data

• Difficulty of monitoring progress

• Duplication of effort

• Underuse of data with absence of key data elements

• Lack of reliable data on inequalities

• Weak capacity to analyze existing data

The NHSP identifies the following strategies and objectives for HMIS:

Strategy 1: Create a harmonized framework for the national health management information system

Objective 1.1 Define key health indicator standards, along with stratifiers (gender, rural/urban, socioeconomic, etc.)

Objective 1.2 Develop platforms for providing essential health information

Objective 1.3 Determine data and analytic capacities

Objective 1.4 Develop frameworks or guidelines for information use

Strategy 2: Strengthen country health information systems

Objective 2.1 Establish collaboration with academic and research Institutions for analytic rigor to make information supplied reliable and acceptable

Strategy 3: Improve access to and use of health information

Health Sector Reform

Health reforms have been implemented in Zambia since 1992 under the framework of the Sector Wide Approach (SWAP). Resources from government and other stakeholders are pooled so they can be used efficiently.

The main success of the health reform processes in Zambia is decentralization down to the district level. In 1996, the Central Board of Health (CBoH) was established as the implementing body. However, failure to implement the delinkage of public health workers from the civil service led to the end of the dualism of the Ministry of Health and Central Board of Health. A new structure of the Ministry of Health is in its final stages of preparation.

It will comprise a monitoring and evaluation unit addressing in a comprehensive way the Health Management Information System.

There was however a change in attitude generated by the new health reforms whereby a number of initiatives (mainly in the private and NGO sector) were seen as being effective.

This showed the possibility of implementing a new national HMIS system which was flexible and more responsive to the needs of all levels of the health system.

Health Management Information System

Before 1994 the Health Management Information System (HMIS) in Zambia was overly centralized and fragmented, and resulted in the collection of large amounts of data which was not useful for decision making and action. A number of problems were identified with the previous system that included fragmentation, duplication, centralization, delay, unreliability.

The system was not producing results and was donor driven, rather than being used for local decision making. Improvement of the HMIS was therefore an important part of overall health reforms and a comprehensive plan was drawn up in 1996 to ensure a functional, district-based

HMIS.

A review of the development and implementation of the 1996-HMIS is presented in Annex 7.

Objectives of the Assignment

The objective of the consultancy is to make an assessment of the existing health management information system in Zambia and to prepare a work plan for its strengthening.

The global objective is to improve the efficiency and effectiveness of health care delivery through the strengthening of HMIS. Support to HMIS is assumed to strengthen poverty reduction monitoring.

The specific objective of the consultancy is to develop a detailed, quantified action plan for a strengthened, revised and comprehensive Health Management Information System, focused on coverage of essential health services and on the health status, but open to integration with other components that may be developed at a later stage.

The assessment follows the following broad areas:

1. The relevance of the currently operational HMIS to the general M&E framework of the health sector.

2. NDP and HSSP information needs from the HMIS

3. Critical information gaps to the NDP monitoring of the Health sector

4. HMIS capabilities in generating gender and age responsive information on key health indicators and the reporting formats and frequency

5. Investment needs for strengthening the HMIS on: infrastructure (hardware and software), and personnel (training, conditions of service, retention ) to operate the

HMIS

6. Mechanisms for quality assurance and control of the HMIS.

This report is a product of an assessment of the health management information system car-ried out by EHG consultants in close collaboration with the Ministry of Health, Central Board of Health and development partners in July-August 2005. The second product of the consul-tancy, a Plan of Action for HMIS strengthening constitutes a separate report.

Methodology

As stated in the Terms of Reference (TOR) the assignment should result in two products,

(i) an assessment of the state of the country health information system, its ability to collect, analyse and generate reliable data on specific indicators, and the extent and timing of data dissemination and use,

(ii) a detailed, quantified action plan that will build the rationale for HMIS reform, the vision to which Zambia should aspire and options and strategies to help Zambia turn that vision into a reality.

It should be quite clear that the content and quality of the action plan, must depend on the results from the assessment of the current HMIS.

The short time allotted to the assessment allowed us only to do a thorough assessment of the HMIS ability to collect, analyse and generate data on specific indicators.

Although we do have some impressions of the quality and the reliability of data, we cannot substantiate or justify our impressions by exact findings. However it is recognized that the quality of the data is an important issue that has to be addressed by the project for strengthening the HMIS.

What follows below is a brief outline of the methodology applied for the assessment of the HMIS, the underlying principles for the exercise, and the key areas of concern.

Consultative and participatory

One of the basic principles that has guided our work is that it must be a consultative and participatory process, involving as many users of the HMIS as possible, from the facility health workers at ground level, through managers at district, provincial, and national level, to external donors and other cooperative partners.

The current HMIS was designed in 1996 and built on the DART-formula, i.e. Decentralised, Action-oriented, Responsive, and Transparent. We have tried to build our assessment using the same principles. The current HMIS has not been revised since its inception. As the ultimate goal of the assignment is to build the rationale for a reform of the existing HMIS we have seen it as an important outcome of the assessment that awareness was raised among the different stakeholders about the need for revising the system.

We consulted as many stakeholders, within and outside the ministry, as practically possible within the timeframe given. We interviewed health staff in all 9 provinces, and in 17 of the 72 districts in the country, and a considerable number of health institutions. We visited rural and urban districts, public, private, semi-public, small health centres with very few staff and the biggest hospital in the country, UTH, with more than 1.600 beds and around 3.000 employees.

During our field visits we were accompanied by staff from the MoH and/or CBoH. When visiting districts and health facilities we were accompanied by the Data Management Specialist from the relevant Provincial and/or District Health Management Board. In that respect the assessment process was also used to facilitate capacity building with in-depth discussions with local staff about findings and the way forward.

Finally, we were aware during all our work that we should be working with and not for the Ministry of Health. We emphasized direct and active participation of the ministry in the exercise, enabling the ministry to take responsibility and ownership of the plan of action that is envisaged as the offshoot of the assessment.

During our work in Lusaka we had daily meetings with the MOH official responsible for the HMIS and a number of officials of MoH and CBoH.

We presented our interim reports at the following meetings:

1. Presentation of Inception Report M&E Meeting 13th July

2. Presentation of 1st Draft of Assessment Report IRS Meeting 29th July

3. Presentation of 2nd Draft of Assessment Report M&E Meeting 03rd Aug.

4. Presentation of Final Draft of Assessment Report IRS Meeting 05th Aug.

Acceptability

The involvement and participation of the ministry in the process of compilation of the report on the assessment was imperative to ensure acceptability and to achieve a realistic plan for strengthening the HMIS.

An effective, reliable and timely HMIS is crucial for ensuring optimal health care delivery and as a tool to strengthen poverty reduction monitoring. Its acceptability depends on its ability to meet local needs and be an effective management and planning tool.

UN Millennium Development Goals (MDG)

Another guiding principle for our assessment has been that the HMIS should monitor the Zambia health sector’s commitment to and achievements in fulfilling the Millennium Development Goals (MDGs).

The United Nations Millennium Declaration is the commitment to making the right to development a reality for everyone and to freeing the entire human race from want. It was signed by 147 heads of state in September 2000. They acknowledged that progress is based on sustainable economic growth, which must focus on the poor, with human rights at the centre.

The objective of the Declaration is to promote "a comprehensive approach and a coordinated strategy, tackling many problems simultaneously across a broad front."

The declaration calls for halving by the year 2015, the number of people who live on less than one dollar a day. This effort also involves finding solutions to hunger, malnutrition and disease, promoting gender equality and the empowerment of women, guaranteeing a basic education for everyone, and supporting the Agenda 21 principles of sustainable development.

To help track progress, the United Nations Secretariat and the specialized agencies of the UN system, as well as IMF, the World Bank and OECD defined a set of time-bound and measurable goals and targets for combating poverty, hunger, disease, illiteracy, environmental degradation and discrimination against women. Each year, the Secretary-General will prepare a report on progress achieved towards implementing the Declaration, based on data on the 48 selected indicators.

Of the 48 selected indicators 15 (i.e. 31,25%) are directly linked to health services (see list of health MDG indicators in Annex 6). Consequently, monitoring and evaluation of health services is a very important part of the monitoring of the achievement in fulfilling the MDG.

HMIS Assessment tool

A tool was developed, based on the WHO HMN assessment framework for assessing the Zambian health information systems (HIS) 1.

The aim of this tool is:

1. To provide a standardised baseline assessment of the country’s health information system

2. To provide a basis for a country HIS development plan,

3. To enable monitoring of progress of the country over time.

Objectives of the tool are:

• Determine performance of sub-systems and overall HIS, and the level of functional integration.

• To assess performance of HIS with respect to data quality, collection, processing, analysis and use of information

• Identify on-going development processes and major problems/shortcomings as well as identifying options for addressing such issues.

• Develop local skills to perform similar assessments in the future

• Provide clear criteria by which country information systems can be judged

This tool provides a standardised baseline assessment for Zambia’s monitoring of poverty reduction as defined in the MDGs. However, the tool also incorporates other important components of health system like for instance legal aspects, infrastructure, data sources and utilisation of available data as illustrated.

1 Unpublished paper: Jorn Braa, Arthur Heywood, Sundeep Sahay, Calle Hedberg: Health Metrics Network. A tool for assessing country health information systems (HIS).

Assessment palette

Monito-ring of MDGs

Health System indicators Legal

Frame-work

Informa-tion

Processes

Human Resources

HIS

Context

Data Sources

The team focused the assessment on indicators for monitoring poverty reduction, and particularly on the MDG indicators. The assessment tool may serve for developing a standardised diagnostic method for monitoring of progress of the Health Management Information System development.

To summarise the methodology, the guiding principles for the assessment have been o Consultative and participatory, o Decentralised, Action-oriented, Responsive, and Transparent, o Acceptable, o Awareness raising, o Capacity building, o Mainly focusing on poverty reduction and MDG indicators

Activities

As illustrated the list of activities in the technical proposal approved by the MoH for the assessment contained 8 steps.

Draft assessment report

National workshop

Final report

Inception

Presentation of inception report

Field visits

Lusaka assessment

Draft plan of action

What follows is a brief description of the activities carried out during the assessment.

1. Inception and Document Review

During the inception and document review, the following activities were carried out:

1. The Terms of Reference (ToR) were clarified and agreed upon with the project steering committee (the MOH Monitoring and Evaluation Subcommittee). The ToR including comments are found in Annex 1.

2. Focus and priority areas for the assessment were defined as the ability of the current HMIS to monitor poverty reduction within the framework of the MDGs.

3. Existing data, reports, studies etc. relating to the HMIS were reviewed and key areas and focus points defined for inclusion in the field assessment (see Annex 7 for details).

4. Instruments and logistics for data collection were agreed on. The general outline of the Health Metrics Network tool was adapted to the realities of Zambia’s health sector. Three different questionnaires were developed, one to be used at national and at provincial level, the second one for district level and the last one for health facilities.

The questionnaires are found in Annex 4.

5. Geographic areas for data collection were agreed upon and the team members allocated to these. One staff member from either MoH or CBoH was appointed to accompany each of the team members on the field work. The composition of the teams and the areas they visited, are found in Annex 8.

6. Finally a plan was elaborated for the logistics of the field visits and the provinces informed about the visits.

2. Presentation of Inception Report

Concurrently with the preparations for the field visits the team was reviewing existing documentation on the design and implementation of the 1996 HMIS plan.

The review was presented in a steering committee, held on Wednesday 13th July. The PowerPoint used as the frame for the presentation is included in Annex 9.

3. Field visits

The field visits took place between 14th and 23rd of July visiting 8 provinces and 16 districts.

Lusaka Provincial and District Health Management boards were visited on 25th July.

Thus, a total of 17 of the 72 districts in the country were visited, which gives a coverage rate of 23.6%. Of the 17 districts visited, 7 were urban, 8 rural, and 2 a mixture of urban and rural.

The sample is seen as representative for the country as a whole.

Type of district No. %

Urban 7 41,2%

Rural 8 47,1%

Mixed urban/rural 2 11,8%

Total districts included in the survey 17 100,0%

Total no. of districts in the country 72

Country coverage rate 23,6%

A total of 33 health facilities were interviewed and included, including public, private, mission, urban and rural health centres, 1st, 2nd and 3rd level hospitals. The table below gives the details.

Type Public % Mission % Private %

1st level hospitals 1 3,0% 1 3,0% 0 0,0%

2nd level hospitals 10 30,3% 5 15,2% 0 0,0%

3rd level hospitals 1 3,0% 0 0,0% 0 0,0%

Urban Health Centres 4 12,1% 0 0,0% 2 6,1%

Rural Health Centres 8 24,2% 0 0,0% 0 0,0%

Hospital Affiliated Health Centres 1 3,0% 0 0,0% 0 0,0%

Total HCs included in the survey 25 75,8% 6 18,2% 2 6,1%

The field visits were spread all over the country, from the densely populated Copperbelt and Lusaka provinces to the remote and sparsely populated provinces in the north, northwest and west of the country.

Results of field visits

As each of the provincial and district questionnaires contained 124 records and the facility level questionnaire 150+ records, the amount of data collected during the field visit was huge.

As it was impossible to analyse this amount of data manually, a database was developed in MS-Access. The database can in the future be utilised to assess the performance of the Zambia HMIS, or other assessments using a similar tool.

The outputs from the database include a number of reports on performance as well as analytical tools that can be used to compare the capacity of the different sub-national users in collection and use of data.

With the database the Zambia health sector (and all other users of the tool) has a tool that can be utilised to continuously assess and evaluate the performance of the HMIS at all levels.

A selected assortment of analytical reports is contained in Annex 5 and the database program engraved on an enclosed CD disc.

The structure of the database follows the HMN framework as illustrated below (see next page and Annex 5).

It should be noted that the assessment tool neither assess health services nor the health information system, but only the health management information system. We are aware that even if the health management information system is running perfectly it does not imply that the health information system is perfect. However, we believe that a well functioning health management system can add value to the health information system and the performance of the health sector.

Database results of the interviews

The database reports on the performance of the health information system for the whole country, for the ministry of health, by province, by district and by facility. Further, the database breaks the data down into 4 levels and in addition reports on all the micro-data collected from the interviews with the health service authorities and providers.

At Level 0 the overall performance of Zambia HMIS by province and MoH is illustrated in the table and graph below. On average the provinces score is 40.7% of the total score, with relative high variations, with Copperbelt as the best province at 55.6%.

Overall performance of Provincial HMB and MoH

0,0%

10,0%

20,0%

30,0%

40,0%

50,0%

60,0%

CEN

COP

EAS

LUA

LUS

NORNOW

SOU

WES

Prov. Av.

MOH

Zambia HMIS performance ratio

Entity Score

CEN 40,7%

COP 55,6%

EAS 47,3%

LUA 26,5%

LUS 44,2%

NOR 26,3%

NOW 39,1%

SOU 43,3%

WES 43,0%

Prov. Av. 40,7%

MOH 48,7%

LEVEL 0 LEVEL 1 LEVEL 2 LEVEL 3

Zambia HMIS

Monitoring of MDG indicators MDG Indicators

Prevalence of underweight children (under five years of age)

Under five mortality rate

Infant mortality rate

Proportion of 1 year old children immunized against measles

Maternal mortality ratio

Proportion of births attended by skilled health personnel

HIV prevalence among 15-24 year old pregnant women

Contraceptive prevalence rate

Prevalence of malaria

Prevalence of tuberculosis

Proportion of TB cases cured under DOTS

Proportion of population with access to affordable essential drugs on a sustainable

Total health expenditure per capitaHealth System Indicators

Monitoring of other indicators

Health workers per 1000 population

Census

Vital registration

Household surveys conducted

Surveillance system for acute diseases

Surveillance system for chronic diseases -

HIV/AIDS/ STI/ TB

Health service statistic s / HMIS

Health systems information

Legal framework and context

Capac ity Building

Legal Framework

Data Sources

Planning and priority settingInformation Processes

Human Resources

HIS context Infrastructure

Data availability and management

Policy advocacy and dissemination

Implementation and action

Tool for assessment of health management information system

At Level 1 the data is broken down to monitor MDG and other indicators. Again the Copperbelt gets the highest scores with 50.0% on MDG indicators and 59.2% on other indicators.

MDG/non-MDG by Provincil HM Board and Min. of Health

0,0%

10,0%

20,0%

30,0%

40,0%

50,0%

60,0%

CEN

COP

EAS

LUA

LUS

NORNOW

SOU

WES

Prov. Av.

MOH

MDG indicators Other indicators

Entity MDG Others

CEN 36,8% 43,2%

COP 50,0% 59,2%

EAS 39,6% 52,2%

LUA 32,6% 22,6%

LUS 42,4% 45,4%

NOR 29,2% 24,6%

NOW 28,5% 45,8%

SOU 44,4% 42,5%

WES 41,7% 43,9%

Prov. Av. 38,3% 42,2%

MOH 38,3% 49,8%

At Level 2 the data from the assessment of monitoring of other indicators is further broken into the 7 HMN-frames as illustrated.

CEN COP EAS LUA LUS NOR NOW SOU WES P. Av. MOH

Monitoring of MDGs 36,8% 50,0% 39,6% 32,6% 42,4% 29,2% 28,5% 44,4% 41,7% 38,3% 46,9%

Health System Indicators 37,5% 33,3% 45,8% 0,0% 0,0% 0,0% 45,8% 66,7% 70,8% 33,3% 27,1%

Data Sources 40,1% 62,5% 46,4% 24,0% 46,4% 25,0% 42,2% 33,3% 35,4% 39,5% 42,7%

Legal Framework 0,0% 66,7% 0,0% 0,0% 0,0% 0,0% 0,0% 33,3% 66,7% 18,5% 0,0%

HIS Context 33,3% 41,7% 50,0% 27,1% 33,3% 20,8% 33,3% 33,3% 37,5% 34,5% 56,3%

Human Resources 52,4% 66,7% 70,2% 31,0% 61,9% 31,0% 50,0% 42,9% 40,5% 49,6% 57,1%

Inform. Processes 53,8% 69,2% 56,4% 23,1% 64,1% 35,9% 61,5% 56,4% 53,8% 52,7% 73,1%

To illustrate the performance a variety of graphs can be applied, looking either

• horizontally (looking at the same systems across all provinces)

• or vertically (looking at all systems in the same province).

The two horizontal examples below show performance on Health Systems Indicators and Data Sources (see Annex 5 for full report). The graphs show considerable variations in the capacity to meet the HMN framework goals with Copperbelt Province performing reasonable well taking in consideration that the HMIS was developed almost 10 years ago and the indicator set has not been revised since then and thus not adapted to demand for monitoring changes.

Health Systems Indicators

37,5%

33,3%

45,8%

0,0% 0,0%

0,0%

45,8% 66,7%

70,8%

33,3%

27,1%

0,0%

10,0%

20,0%

30,0%

40,0%

50,0%

60,0%

70,0%

CEN

COP

EAS

LUA

LUS

NORNOW

SOU

WES

Average

MOH

Data Sources

40,1%

62,5%

46,4%

24,0%

46,4% 25,0%

42,2%

33,3%

35,4%

39,5%

42,7%

0,0%

10,0%

20,0%

30,0%

40,0%

50,0%

60,0%

70,0%

CEN

COP

EAS

LUA

LUS

NORNOW

SOU

WES

Average

MOH

Looking at the results from a vertical point of view, the performance of each province broken down into the elements of the HMN framework may be illustrated. The three examples below are Central, Luapula, and Western Province, respectively.

Central Province

36,8%

37,5%

40,1% 0,0%

33,3%

52,4%

53,8%

0,0%

10,0%

20,0%

30,0%

40,0%

50,0%

60,0%

70,0% Monitoring of MDGs

Health System Indica

Data Sou

Legal FrameworkHIS Context

Resources formation Processes

Luapula Province

32,6%

0,0%

24,0%0,0%

27,1%

31,0%

23,1%

0,0%

10,0%

20,0%

30,0%

40,0%

50,0%

60,0%

70,0% Monitoring of MDGs

Health System Indicato

Data Sources

Legal FrameworkHIS Context an Resources nformation Processes

Western Province

41,7% 70,8%

35,4%

66,7%

37,5%

40,5%

53,8%

0,0%

10,0%

20,0%

30,0%

40,0%

50,0%

60,0%

70,0% Monitoring of MDGs

Health System Indicators

Data Sources

Legal FrameworkHIS Context man Resources

Information Processes

Again, there are considerable variations in how well the provinces perform in relation to the seven groups of indicators. It should be taken into consideration, though, that these graphs do not illustrate the performance of provinces in relation to each other.

The final graph illustrates the performance of the national level, i.e. the combined MoH and CBoH.

Central Level (MoH & CBoH)

46,9%

27,1%

42,7% 0,0%

56,3%

57,1%

73,1%

0,0%

10,0%

20,0%

30,0%

40,0%

50,0%

60,0%

70,0%

80,0% Monitoring of MDGs

Health System Indicators

Data Sources

Legal FrameworkHIS Context uman Resources

Information Processes

Findings

Millennium development Goals

MDGs are generally not regularly assessed in the Zambian HMIS. Much data is collected that could potentially be used to analyze the MDGs; however, much of that data is not used optimally and many of the indicators are not calculated by provincial or district managers.

The following table presents the internationally agreed MDGs and comments on how each one is measured and interpreted in Zambia.

MDG Indicator Zambian Measurement

MDG Indicator 4: Prevalence of underweight children under five years of age MDG Goal 1: Eradicate extreme poverty and hunger MDG Target 2: Halve, between 1990 and 2015, the proportion of people who suffer from hunger

• Indicator is measured quarterly during performance assessment using routine data.

• Disaggregated by facility but not by gender or more detailed age group

• No record review or surveys

• No reflection on whether the available facility data represents the whole population

MDG Indicator 13: Under five mortality rate MDG Goal 4: Reduce child mortality MDG Target 4: Reduce by two third, between 1990 and 2015, the under five mortality rate

• Not done (except DHS), though Lusaka is starting to look at some facilities.

• Data is institution based; Death registration is negligible

• DHS and other survey results are generally not known

MDG Indicator 14: Infant mortality rate MDG Goal 4: Reduce child mortality MDG Target 4: Reduce by two third, between 1990 and 2015, the under five mortality rate

• Not done.

• Data is only institution based; death registration is negligible

• DHS and other survey data generally not known MDG Indicator 15: Proportion of 1 year old children immunized against measles MDG Goal 4: Reduce child mortality MDG Target 4: Reduce by two third, between 1990 and 2015, the under five mortality rate

• Indicator measured quarterly during performance assessment using routine data.

• Disaggregated by facility but not by gender or more detailed age group

• No quality control by record review or surveys

• Includes private data in Lusaka and Copperbelt because vaccines are given free

MDG Indicator 16: Maternal mortality ratio MDG Goal 5: Improve maternal health MDG Target 6: Reduce by three-quarters, between 1990 and 2015, the maternal mortality ratio

• Not calculated, though institutional maternal deaths and live births are available.

• Some districts and provinces do maternal death audits but generally do not relate to live births,

• Maternal Death registration negligible,

• Cause, time before death and age are not calculated,

• DHS and other survey results not known.

MDG Indicator 17: Proportion of births attended by skilled health personnel MDG Goal 5: Improve maternal health MDG Target 6: Reduce by three-quarters, between 1990 and 2015, the maternal mortality ratio

• Indicator measured quarterly during performance assessment using routine data.

• Disaggregated by facility but not by more detailed age group or type of personnel (TBAs, CBEs or midwives)

• No record review or surveys

• No reflection on whether facility data represents the whole population MDG Indicator 18: HIV prevalence among 15-24 year old pregnant women MDG Goal 6: Combat HIV/AIDS, malaria and other diseases MDG Target 7: Have halted, by 2015 and begun to reverse, the incidence of malaria and other major diseases

• Not done at any level and results of national surveys generally unknown,

• Data available at some laboratories but not analysed by this age group,

• No use of sentinel sites to get this indicator … data does not go to provinces,

• The number of 15-24 year old pregnant women not generally known.

MDG Indicator 19: Contraceptive prevalence rate MDG Goal 6: Combat HIV/AIDS, malaria and other diseases MDG Target 7: Have halted, by 2015 and begun to reverse, the spread of

HIV/AIDS

• Not done at any level – only “New acceptors”, which is a meaningless indicator,

• Data available on number of each type of contraceptive distributed, so one could easily calculate couple year protection, which is a proxy indicator of CPR,

• DHS results unknown.

MDG Indicator 21: Prevalence of malaria MDG Goal 6: Combat HIV/AIDS, malaria and other diseases MDG Target 8: Have halted, by 2015 and begun to reverse, the incidence of malaria and other major diseases

• Incidence at facilities is measured, NOT prevalence in communities,

• Done on a quarterly basis, using routine facility based data,

• Some districts doing surveys run by National level but data not at provinces,

• Quality of data questionable because of IMCI “syndromic” approach (fever = malaria),

• Minimal use of sentinel site data and record reviews,

• No reflection on whether data from the health facilities represents the whole population.

MDG Indicator 23: Prevalence of tuberculosis MDG Goal 6: Combat HIV/AIDS, malaria and other diseases MDG Target 8: Have halted, by 2015 and begun to reverse, the incidence of

• Incidence at facilities is measured, NOT prevalence in communities,

• Quarterly analysis of routine data in all districts,

• Little further breakdown of incidence by malaria and other major diseases gender and age, though data is there.

MDG Indicator 24: Proportion of TB cases cured under DOTS MDG Goal 6: Combat HIV/AIDS, malaria and other diseases MDG Target 8: Have halted, by 2015 and begun to reverse, the incidence of malaria and other major diseases

• Not regularly analysed … most respondents claim that ALL TB cures are on DOTS so there is no need to differentiate them from others,

• Problem with cohort system – up to 60% of patients on DOTS do not have initial sputum, so not in “cured” cohort,

• Data not integrated into routine HMIS so provinces not regularly informed.

MDG Indicator 46: Proportion of population with access to affordable essential drugs on a sustainable basis MDG Goal 8: Develop a Global partnership for development MDG Target 8: provide access to affordable essential drugs in developing countries

• Not analysed anywhere,

• Analysis of drug packs opened per patient (utilisation) and out of stock per facility (availability) relates to service provision but do not address per capita issues.

National MDG progress report

There was an assessment of MDGs for the whole country1 to compare data from 1990 and 2002.

The following results are given:

Indicator 1990 2002 Probability of meeting target

% underweight 25 28 Unlikely

Under five mortality 191 163 Potentially

Infant Mortality 107 95 Potentially

<1 measles coverage 77 84 Potentially

Maternal Mortality 649 729 Unlikely

Skilled Birth Attendances 51 45 Unlikely

ANC HIV positive 15-24yrs 19 17 Potentially

Malaria new cases (NOT prevalence) 255 377 Potentially

There are great gaps in the data and no mention is made of TB prevalence or DOTS, drug availability or CPR. While no mention is made of the precise data sources, the results come mainly from the ZDHS, Annual Health statistical bulletins, ANC sentinel surveillance and Roll Back Malaria surveys.

A sustained ongoing assessment of MDGs from health facilities at the periphery using routine data and record review would undoubtedly give important support to improving the reliability and coverage of these figures and make MDGs more real for front-line service providers.

1 MDG progress report 2003 by GRZ and UNDP

Health system Indicators

Data for these health system indicators are generally available, but they are not calculated because there is no demand for them in the HMIS and it is difficult to calculate them with the current database.

Total health expenditure per capita • Not analysed anywhere though figures are generally available,

• FAMS not integrated into routine HMIS,

• Analysis on Health budget by facility not done.

Health workers per 1000 population (includes physicians, nurses, midwives)

• Not analysed anywhere, though figures are said to be available from the HR-registers,

• Reported on the basis of health workers per patient (workload),

• Data available on health workers per facility (accessibility),

• Generally no reflection on the reliability of the data based on projections from

CSO.

Population Data

The census report is generally available only at provincial level, and even there it was usually not electronically available, easily accessed or much used. Provincial managers claim that the census report has been distributed to the district but it was rarely found at the districts visited.

The projections seen are not disaggregated by gender and age. At district level there are projections of population by district and by facility and these are generally used as a first line for estimation of coverage of services.

There is not much faith in these census figures due to poor counting by enumerators and many provinces and districts use local headcounts for these populations, in order to avoid very high…