Attachment_J.16_M E_Strategic_Plan_UG_2010.11-2014.15.pdf

GOVERNMENT OF UGANDA

Ministry of Health

MONITORING & EVALUATION PLAN

FOR

NATIONAL MALARIA CONTROL

STRATEGIC PLAN 2010/11 – 2014/15

Plot 6 Lourdel Road, Wandegeya P. O. Box 7272, Kampala, Uganda

December 2011

Any part of this document may be freely reviewed, quoted, reproduced or translated in full or in part, provided the source is acknowledged. It may not be sold or used in conjunction with commercial purposes or for profit.

Government of Uganda, Ministry of Health: Monitoring & Evaluation Plan for National Malaria Control Strategic Plan 2010/11 – 2014/15

Published by: Ministry of Health PO Box 7272 Kampala, Uganda Email: info@health.go.ug Website: www.health.go.ug i | P a g e mailto:info@health.go.ug http://www.health.go.ug/

Foreword

By 2015, Uganda aims to have significantly reduced malaria in the country. It is therefore imperative that focused plans and interventions are monitored regularly and evaluated periodically to access progress, shortcomings and inform the Ministry of Health (MoH) on the next steps. A sound monitoring and evaluation (M&E) framework is critical for the success of malaria program implementation. An appropriate performance framework can demonstrate outcomes and impact achieved in prevention and control of malaria.

This M&E plan has been written to ensure that indicators, their definitions and means of data collection and measurement are comparable over time. It is also meant to reduce duplication of efforts by both partners and MoH through ensuring the core principle of the

Roll Back Malaria (RBM) partnership. In addition, the purpose of this M&E plan is to monitor and evaluate the strategic plan for the National Malaria Control Program (NMCP) -

2011/12 to 2014/15. The plan also identifies desired data sources and roles of key malaria

M&E stakeholders in the country.

Dr. Jane Ruth Aceng, Director General of Health Services ii | P a g e

Acknowledgements

The Ministry would like to thank Dr. Sarah Byakika, Assistant Commissioner Quality

Assurance Department, and Dr. Eddy Mukoyo, Assistant Commissioner, Resource Centre at

MoH for policy and technical guidance to the development of this M&E plan.

Special thanks also go to the RBM Partners, United States President’s Malaria Initiative

(PMI), Management Sciences for Health (MSH), the Global Fund to Fight Aids Tuberculosis

(Global Fund), and the World Health Organization (WHO) Country Office for their commitment, technical support and overall participation in the process.

I would like to thank the National Malaria Control Program Manager, Dr. Seraphine Adibaku, for providing leadership for this exercise and thank his team for the strong work.

Finally, I would like to express my sincere gratitude to PMI through the Stop Malaria Project for financing the development of this M&E plan.

Dr. Asuman Lukwago, Permanent Secretary iii | P a g e

Acronyms

ACT Artemesinin Combination Therapy

ADR Adverse drug Reactions

AMFM Affordable Medicines Facility for malaria

ANC Antenatal Care

AWP Annual Work plan

BFP Budget Framework Papers

CBO Community Based Organisation

CMD Community Medicine Distributor

CSO Civil Society Organisation

DHIS Digitalised Health Information System

DHO District Health Officer

DQA Data Quality Audit

DSS Demographic Surveillance Site

EPI Expanded Program on Immunisation

EPR Epidemic Preparedness and Response

GF Global Fund

HF Health Facility

HMIS Health Management Information System

HPAC Health Policy Advisory Committee

HSD Health Sub District

HSS Health System Strengthening

HSWG Health Sector Working Group

HW Health Worker

ICCM Integrated Community Case Management

IDSR Integrated Disease Surveillance and Response

IPT Intermittent Presumptive Treatment of Malaria during Pregnancy

IRS Indoor Residual Spraying

ITN Insecticide Treated Net

IVM Integrated Vector Management

JRM Joint Review Mission

LG Local Government

LLIN Long Lasting Insecticide Treated Net

M&E Monitoring and Evaluation

MESST Monitoring and Evaluation Systems Strengthening Tool

MOH Ministry of Health

MOPS Ministry of Public Service

MPR Malaria Program Review

NHA National Health Assembly

NHP Nation Health Policy

NHSSIP National Health Sector Strategic and Investment Plan

NIP National Investment Plan

NMCP National Malaria Control Program

OPD Outpatient Department

PHP Private Health Practitioners

PMI U.S. President’s Malaria Initiative

PNFP Private Not for Profit

PSM Procurement and Supplies Management

RBM Roll Back Malaria

RC Resource Centre

RDT Rapid Diagnostic Test

SBCC Social Behaviour Change Communication

SMP Stop Malaria Project

SMS Short Message Service

SP Sulphadoxine/Pyrimethamine Tablets

SWAPS Sector Wide Approaches

TWG Technical Working Group

UBOS Uganda Bureau of Statistics

UCP Uganda Capacity Program

UDHS Uganda demographic Health Survey

UMSP Uganda Malaria Surveillance Project

USAID United States Agency for International Development

VHT Village Health Team iv | P a g e

Table of Contents

1 INTRODUCTION

1.1 Background

1.2 Epidemiology

1.3 Policy Environment

1.4 Organization of the National Health System

1.5 Organisation of NMCP

2 Monitoring and Evaluation Implementation Approach

2.1 Strategic Direction for Monitoring and Evaluation Strategic Plan

2.2 The Role of M&E Unit

2.3 Role of Partners in Achieving One M&E Plan

2.4 Reporting options to ensure targets and indicators are harmonized

2.5 Existing quality assurance systems

2.6 Sector-wide approach and reporting mechanisms among partners

2.7 Plans to strengthen M&E in the MoH

2.8 Achievements in Malaria M&E strengthening

2.9 Summary of the National Malaria Strategic Plan

3 Strategic Framework for M&E Plan

3.1 Broad Objective:

3.2 Objectives:

3.3 Monitoring, Evaluation and Reporting Framework

3.4 Key Indicators

4 Data Collection and Reporting

4.1 Data collection methods

4.2 Data sources

4.3 Timeframe for M&E Review

5 M&E Implementation Arrangements

5.1 National Context

5.2 Coordination of malaria M&E

5.3 Malaria M&E within the NMCP

5.4 Monitoring the Implementation of the NMCP M&E Plan

5.5 Evaluation of the Malaria Strategic Plan (MSP)

5.6 Data Flow

5.7 Data Quality Assurance and verification

5.8 Malaria Database

5.9 RBM Mechanism for Review and Action

5.10 Operational Research and Special Studies

5.11 Dissemination of the information and information products

6 Action for Implementing the M&E Strategic Plan 7 Malaria Monitoring and Evaluation Framework v | P a g e

1 INTRODUCTION

1.1 BACKGROUND

Malaria is preventable and treatable, yet approximately 70 to 100 Ugandans die daily from malaria. Malaria is the highest ranked cause of morbidity in Uganda with over 30 million cases estimated to occur each year in the country. The disease kills more children under the age of five in Sub-Saharan Africa than any other single disease, and it is a major cause of complications, including death, in pregnant women.

1.2 EPIDEMIOLOGY

Clinically diagnosed malaria is the leading cause of morbidity and mortality, accounting for 25- 40% of outpatient visits at health facilities, 15-20% of all hospital admissions, and 9-14% of all hospital deaths. Nearly half of inpatient deaths among children under five years of age are attributed to clinical malaria. A significant percentage of deaths occur at home and are not reported by the facility-based Health Management Information System (HMIS).

Malaria is highly endemic in 95% of the country, covering approximately 90% of the population of 31 million. The remaining 5% of the country consists of unstable and epidemic-prone transmission areas in the highlands of the south- and mid-west, along the eastern border with Kenya, and the Northeast border with Sudan. In some areas of northern Uganda, the entomological inoculation rates (infective biting rates by the mosquitoes that transmit malaria) are among the highest recorded in the world. The Malaria Indicator Survey conducted in 2009 reported high prevalence of malaria parasites in children <5 years of age ranging from 5% in Kampala to 63% in mid northern region, with a national average of 45%. See Figure 1 above.

1.3 POLICY ENVIRONMENT

The period covered by this plan is most important in the history of malaria control in the country, as it is a period for achieving not only Universal coverage for all interventions, but also consolidation of this status once achieved as well as malaria partnership to facilitate extensive internal and external mobilization for the resources needed to enable and sustain this massive scaling up.

The Malaria Program has remained as a priority action within the national health agenda in Uganda. The Political Presidential Manifesto 2010 points towards “Malaria elimination” in five years. The Constitution of the Republic of Uganda, National Investment Plan (NIP 2010/11- 2014/15), and the NHSSIP III position malaria among the programs of national interest

Figure 1: Endemicity Map

The National Health Policy (NHP) and the National Health Sector Investment Plan (2010-15) are implemented through partnerships within the broad framework of the Health Sector Wide Approaches (SWAps). Existing mechanisms such as Health Policy Advisory Committee (HPAC), bi-annual joint review missions (JRM), Health Sector Working Group (HSWG) and annual National Health Assembly (NHA) ensure coordination, continuity and regular reviews within the health sector. The country has developed a new comprehensive malaria policy, which will guide all malaria interventions in the country.

A number of policy supports towards ensuring that malaria is controlled in the country have been put in place. User fees were abolished in all public facilities and malaria treatment is free and government taxes on ITNs and medicines are waived. Government has embraced AMFm, which will ensure that effective antimalarials are available at an affordable price in private outlets.

1.4 ORGANIZATION OF THE NATIONAL HEALTH SYSTEM

The National Health System (NHS) is made up of the public and the private sectors. The public sector includes all GoU health facilities under the MoH, health services of the Ministries of Defence (Army), Education, Internal Affairs (Police and Prisons) and Ministry of Local Government (MoLG). The private health delivery system consists of Private Not for Profit (PNFPs) providers, Private Health Practitioners (PHPs), and the Traditional and Complementary Medicine Practitioners (TCMPs).

The formal health care system in Uganda is stratified into the National and Regional referral Hospitals (NRH & RRH), with a five-tier system at District level consisting of the General Hospitals (HC V), health centre IV (Health Sub-district), health centre III (Sub-county), health centre II (Parish) and the health centre I at village level. In each of the 112 districts the District Health Officer (DHO) is responsible for overseeing all facilities (including pharmacies and drug shops) and health services in the district, including those operated by not-for-profit organizations, partners and the private sector. Some responsibilities are delegated to the Health Sub-Districts that form the lower level of health services management. Although not a physical structure, the Health Centre I is at community level organized in “village health teams”. The traditional and complementary medicine practitioners are organized in several professional organizations and play an important role in malaria control in Uganda.

1.5 ORGANISATION OF NMCP

The NMCP is in the National Communicable Disease Control Division, which is a division in the Directorate of Disease Control of the MoH. As a national program, it is mandated to coordinate and steer malaria control activities at national and local government, facility and community levels whether provided by public or private providers, and other partners.

The mandate of the NMCP is to guide the implementation of national malaria control policies to reduce the malaria burden in Uganda. It is therefore expected to exercise leadership and authority in bringing malaria

Figure 2: NMCP M&E Coordination Mechanism under control to a point that it no longer constitutes a public health problem in the country through strategic actions.

The NMCP has the linkages with other departments in the MoH, other national agencies and national partnership structures within the RBM partnership. Within the RBM partnership, the M&E unit of NMCP works through the M&E and Research Coordination Technical Working group shown in Figure 2.

2 MONITORING AND EVALUATION IMPLEMENTATION APPROACH

2.1 STRATEGIC DIRECTION FOR MONITORING AND EVALUATION STRATEGIC PLAN

As the country embarks on ambitious targets for malaria control, sound monitoring and evaluation of performance and associated impact on malaria burden is essential to guide the interventions carried out within the RBM partnership. It is also important for the NMCP to coordinate partner M&E and define the essential M&E roles necessary for understanding progress in attaining the national targets.

This plan recognizes that the national strategic direction for rapid scale up imparts more demands on M&E implementation especially regarding (1) increasing emphasis of data collection on coverage and quality of services (2) generating more detailed information on specific outcome and impact indicators (3) monitoring absorption capacity and other critical service delivery support systems (4) refining epidemic detection. This calls for use of standardized measurement instruments across all partners and levels and to strengthen linkages with the Resource Centre to enhance quality of data and its analysis across technical (e.g. IRS, case management, LLIN) and support (e.g. commodities, human resource) interventions.

2.2 THE ROLE OF M&E UNIT

The role of the M&E unit will be enhanced to cover weaknesses identified in the MPR and the MESST. The unit will work very closely with the Resource Centre and Quality Assurance Department in MoH to ensure that the malaria database is functional and accessible, and engage a biostatistician to improve data analysis and presentation. All data from facility and non-facility based interventions will be deposited in the data warehouse within the Resource Centre.

The unit will also spearhead the revitalization of the M&E and research technical working group of the RBM partnership to ensure conformity to the “one M&E plan” system.

The key M&E plan implementation tasks are:

a. Producing monthly, quarterly and annual results-oriented reports, linked to the NMCP strategic and annual plans.

b. Ensuring that M&E Units assign one or more positions responsible for statistical production, monitoring and evaluation

c. Ensuring that malaria data collection systems and tools are in place and functioning

d. Developing sub-national (regions, districts, HSDs) M&E capacity

e. Planning and budgeting for monitoring and evaluation annually

f. Ensuring that RBM meetings have quality actionable performance review reports.

g. Providing quarterly data and explanatory information on progress against performance indicators to NMCP, RBM, GF, and PMI etc.

h. Ensuring proper coordination and oversight of M&E activities among the partners in relation to the NMCP strategic, annual plans and guidance from M&E TWG.

i. Planning and budgeting for evaluations of all partner projects and programs

j. Utilizing M&E findings to inform NMCP policy, and resource allocation decisions.

k. Quality Assurance through monitoring of quality of service delivery, client satisfaction and tracking of training processes and quality through training information systems

l. Maintaining a recommendation implementation tracking plan, which tracks review and evaluation recommendations, agreed follow-up actions, and status of these actions.

m. Ensuring that complete and approved M&E reports and updated statistical data are made easily available to partners and the public in a timely manner, while ensuring that the sharing of reports respects the access to Information Act.

2.3 ROLE OF PARTNERS IN ACHIEVING ONE M&E PLAN

The M&E plan will be implemented within the framework of mandates in the national health system and calls for a transparent and documented process to ensure input of a broad range of stakeholders in the NMCP monitoring and evaluation. Operationalization of this M&E plan will involve institutions at various levels of the health sector as outlined below.

a. Central Level: This level will be coordinated through the RBM partnership to which all partners, stakeholders and related sectors operating at national level will report.

b. Regional Level: The MoH is in the process of defining this level to provide coordination of monitoring and evaluation in regions, liaise between national level, and the districts on M&E, support development, implementation, and review of the M&E plans of the districts, and supporting operational research.

c. Local Government Level: This level will be coordinated by the district health office and will capture all M&E components (public and non government) in districts, sub-counties, urban councils and private sector based at district level. Data here shall be all facility based and non-facility based malaria activities and should tally with what is presented at national level.

d. Community Level: This level includes the LC III councils (sub-county level), parishes, village councils, private sector, CSOs and CBOs. This data should enter the district level.

To this effect, all partners are expected to work within one national M&E plan to measure their progress and assess impact. To achieve the one M&E Plan, all partners will need to perform the following:

• Align their project objectives and activities with National Malaria Strategic Plan and select appropriate indicators in the M&E plan they contribute to.

• Facilitate harmony between the NMCP and partner M&E units.

• Streamline data flow systems between partners and districts and NMCP.

• Ensure qualified staff are employed and facilitate capacity building on data management and support supervision for quality assurance; as needed.

• Build behaviours, relationships and values towards one M&E plan through ensuring clarity, commonality and commitment to M&E purpose.

2.4 REPORTING OPTIONS TO ENSURE TARGETS AND INDICATORS ARE HARMONIZED

The country-led platform for monitoring and reviews of the health sector are laid out in the HSSIP 2010/11 – 2014/15 M&E plan and this serves as the basis for all M&E related processes and reporting options for NMCP. Figure 3 shows the common M&E platform for NMCP strategic plan with country data generation and use processes in the centre.

Global reporting requirements are based on on-going country processes of data generation, compilation, analysis and synthesis, communication and use for decision making as spelled out in the country compact for implementation of the HSSIP. Options for reporting on malaria data exist and relate to international targets. As a member of the global malaria community, the NMCP is mandated to report into the WHO malaria database that ranks the country on the world malaria scale. Other global requirements include reporting to the Global Fund and RBM.

Reporting at national level is in alignment with the national investment plan (NIP), joint assessment framework, HSSIP and specific program/project requirements as stipulated in the M&E Plan for the HSSIP. At the district level, reporting is based on the HMIS, which is the main source of statistical data for the health sector. The HMIS system captures weekly surveillance disease data (form 033b), monthly outpatient attendance (form 105) and inpatient data (form

Figure 3: Country-led platform for monitoring & review of the NMCP strategic plan (WHO)

108), and is supposed to capture community level (VHT) data as well. The HMIS integrates critical malaria indicators such as, number of malaria cases treated, cases parasitologically diagnosed, IPTp, ITNs distributed at EPI and ANCs. Most of the data reported is generated from public and PNFP health facilities, but data is missing from the private, for-profit facilities (PHP) where about 60% of malaria cases are estimated to seek treatment. The Resource Center plans to streamline reporting from all stakeholders by extending the HMIS to PHP health providers who will be reporting through the DHO for onward transmission of their facility data to the resource centre at MoH.

2.5 EXISTING QUALITY ASSURANCE SYSTEMS

Since its inception in the NMCP, the M&E unit has not been in position to put data quality assurance (DQA) safeguards in place, due to inadequate funding (funding for the M&E unit, which is less than 1% of the NMCP government supported budget). However, given the fact that the unit has global obligations to bodies like RBM and GFATM, efforts to submit quality data have been made. Through the Resource Centre, HMIS data from health facilities is usually crosschecked during area support supervision from the national level. It is also presumed that district quarterly support supervisions do take place with subsequent HSD supervision of lower HFs within the district. This approach somehow ensures elements that cross check the data ultimately transmitted to the RC at the levels in question. Community malaria data, mainly captured by the VHT, has not been fully embraced in the whole country, thus leaving room on the quality of data submitted to the district from this team.

NMCP does not yet check the data quality from partners.

2.6 SECTOR-WIDE APPROACH AND REPORTING MECHANISMS AMONG PARTNERS

This plan is designed to harness the contribution of partners and other key stakeholders through a coordinated mechanism that not only captures data from partners but also facilitates platforms for reporting. Through regular meetings as stipulated in the AWP 2011/12, reporting mechanisms will be enhanced to enable NMCP to meet national, regional and international targets.

2.7 PLANS TO STRENGTHEN M&E IN THE MOH

The HMIS at the MOH has been strengthened to expedite timeliness and correctness of the data submitted to the Resource Centre as a way of improving quality. Efforts by the Resource Centre to increase the number of health facilities and other sources of malaria from the current 85 to 100% have also been put in place. The MOH is embracing technological tracking of key variables from points of health care in the public sector through M-Track. The Intelligent Integrated Computer System project is being piloted at the National Referral Hospital with plans to scale it up to regional referral hospitals and other HFs subsequently. The DHIS 2 electronic software tool is also being piloted by the RC for HMIS data management at all levels of healthcare in the public sector. These approaches, plus other non-conventional methods that include, inter alia telephone calls are employed to address timeliness of data sent to the program.

2.8 ACHIEVEMENTS IN MALARIA M&E STRENGTHENING

Until 2008, the NMCP did not have a formalized M&E Plan. The NMCP M&E Plan 2008-2010 followed realization for the need to have a tool for monitoring and evaluating contributions and effort made towards malaria control in the country. Like all plans, the NMCP M&E Plan 2008-

2010 was built to fit into the remaining period (2008-2010) of the National Malaria Control Program Strategic Plan ending 2010. During its implementation period, substantial achievements were made.

A Malaria Indicator Survey was conducted in 2009 to provide the overall malaria situation in the country. The findings indicated that malaria is still a major problem as evidenced by a parasite prevalence in the under fives of 63% in northern Uganda and 12% in areas hitherto known to be hypo endemic in south western Uganda, with the national average being at 43%.

Evaluation of the NMCP M&E component was conducted using the MESST 2009, and MESST 2011. A Malaria Program Progress Review for the period 2001 to 2010 was carried out and completed in July 2011. All these evaluations identified strengths, weaknesses, opportunities and threats in the M&E component.

To build capacity for the M&E unit, technical assistance (TA) was provided by GFATM and PMI (through Stop Malaria Project). A data management officer was also recruited. A Rapid SMS system was piloted in the districts of Kabale and Gulu. This tracks ACT stock out at health facility level. A similar system (m-TRACK) is scaling up this concept in 2011/2012.

2.9 SUMMARY OF THE NATIONAL MALARIA STRATEGIC PLAN

This M&E Plan works within the NMCP Strategic Plan 2010/11 – 2014/15 whose strategic direction is to rapidly scale-up interventions to universal coverage, achieve consolidated control and set the ground for pre-elimination in the next strategic plan period.

Table 1: National Malaria strategic Framework Vision: A Malaria free Uganda

Mission: • “To provide to all people in Uganda quality services for malaria prevention and treatment”

Goals: • To reduce morbidity due to malaria by 75% of 2010 level’s

• To reduce mortality due to malaria by 80% of 2010 levels.

Objective: Key Intervention Objective 1: To reduce malaria prevalence by 75% of 2010 levels by 2015

• Reach universal coverage and utilization of Long-Lasting Insecticidal Nets (LLINs) so that each household owns at least one LLIN for every two persons

• Scaling up of routine Indoor Residual Spraying to ensure that interior walls of targeted institutional and domestic structures in each district are routinely sprayed at appropriate intervals with an effective insecticide

• Application of chemical and biological larvicides

• Integration of environmental management control measures into on-going public and private sector activities

• Application of live-bait technology

• Capacity building for implementation and coordination of IVM at all levels

• Establish quality assurance on LLINs and IRS, insecticide resistance monitoring, and studies on vector behaviour

• Adoption and implementation of effective policies for timely procurement of necessary commodities for IVM Objective 2: To increase to 90% by 2015 the proportion of suspected malaria cases confirmed and treated with effective antimalarials

• Scale-up quality parasitological diagnosis with microscopy and RDTs

• Appropriate treatment of malaria at public and private health facilities

• Scale up home management of malaria (including referral) within ICCM strategy

• Strengthen capacity for pre-referral treatment and management of severe malaria

Objective 3: To achieve by 2015, 80% of the population consistently using at least one malaria prevention method together with appropriate treatment seeking behaviours

• Advocacy in all political, economic and social spheres;

• Empowerment of women in malaria control activities;

• Community mobilization to change beliefs, attitudes and practices towards malaria treatment and prevention;

• Strengthen alliances with CSO and private sector for SBCC strategic planning;

• Research on behavioural obstacles to poor uptake of interventions;

• Expand effective engagement with business coalitions in malaria Control.

Objective 4: To strengthen M&E systems to assess progress towards set targets, and informing refinement and decision making during implementation;

• Strengthen the functionality of the national RBM M&E working group

• Strengthen the NMCP M&E Unit, NMCP and partner reporting system to monitor the strategic plan

• Strengthen District M&E capacity and functionality encompassing logistics/inventory monitoring, private sector, community VHT reporting

• Health facility based sentinel malaria surveillance including in-patient reporting

• Strengthen linkage between research and other sectors with NMCP

• Monitoring human resource capacity for malaria control

• Quality Assurance through monitoring of quality of service delivery, client satisfaction and tracking of training information systems

• Strengthen monitoring and evaluation of community BCC activities and IVM

Objective 5: To strengthen NMCP for effective malaria control policy development, planning, management, partnership coordination and timely implementation of planned interventions in order to achieve all country objectives and targets set for 2015.

• Advocacy and resource mobilization

• Ensure well-coordinated efforts to scale up

• Elevate NMCP to the level of a Department in the MoH

• Strengthening competencies within malaria control related sectors

• Strengthening PSM systems for malaria commodities

• Strengthening Human Resource capacities for malaria control

• Establishing performance improvement at district level.

3 STRATEGIC FRAMEWORK FOR M&E PLAN

3.1 BROAD OBJECTIVE:

To provide a joint framework for a well coordinated, systematic and holistic tracking of progress in malaria control, informing refinement and guiding decision-making for program improvement.

3.2 OBJECTIVES:

1. To describe the types of data and data sources, and how data will flow from the source to a central repository and to all relevant stakeholders.

2. To provide standard indicators, targets and frequency of reporting in a standardized format for all malaria implementers and stakeholders.

3. To guide the routine and periodic documentation of planned activities and measure expected outputs, and outcomes and impact when due.

4. To define implementation arrangements with clear responsibility centres.

5. To identify and cost key actions that will enable smooth implementation of the plan.

3.3 MONITORING, EVALUATION AND REPORTING FRAMEWORK

The monitoring framework for tracking progress is informed by the need to comprehensively monitor, and review malaria within the health sector progress. The framework for the analysis is based on (M&E) logical framework, which shows the way in which inputs may lead to desirable impact. The framework is intended to ensure that all indicator areas - from inputs to impact - are considered in the analysis, and pathways of influence are clarified (see Figure 4).

Figure 4: Sector M&E Framework

3.4 KEY INDICATORS

In order to monitor the performance of the NMCP in the country, all partners will use the following indicators. Detailed tables of definitions, including inputs and processes, are shown in Annex 1.

Table 1: List of Core Indicators for Malaria in Uganda

A. IMPACT INDICATORS

Malaria deaths:

Under five, all-cause child mortality

Malaria transmission:

Proportion of children under five (6–59 months) with malaria parasites (Parasite prevalence)

Malaria cases:

Malaria cases (per 1,000 persons per year)

Confirmed malaria cases (microscopy or RDT) per 1,000 persons per year

Inpatient malaria cases (per 1,000 persons per year) Proportion of children 6–59 months old with moderate or severe anaemia

Percentage of OPD visits attributed to malaria

B. OUTCOME INDICATORS

Insecticide Treated Nets

“Administrative” ITN coverage Proportion of households with at least 1 ITN

Proportion of households with at least 2 ITNs Proportion of households reaching universal coverage with ITNs (1 net/2 people)

Indoor Residual Spraying

Proportion of targeted population protected by IRS Proportion of households reporting that their household was sprayed with a residual insecticide in the last 12 months

Intermittent Preventive Treatment (IPT) for Pregnant Women Proportion of women who have received two or more doses of IPTp during their last pregnancy in the last two years

Proportion of health facilities with no stock outs of recommended drug for IPTp during the last one month

Diagnosis

Proportion of suspected malaria cases that are tested by either microscopy or RDT Proportion of clinical malaria cases that are confirmed by microscopy or RDT at health facility level

Proportion of suspected malaria cases confirmed to be positive at health facility level (malaria test positivity rate) (disaggregated by age under 5 and above five)

Proportion of fever cases at community level that received an RDT

Case Management

Percentage of outpatient malaria cases that received an appropriate antimalarial treatment according to national policy (disaggregate by age)

Proportion of health facilities with no stock outs of first line anti-malarial drugs during the last one month

Proportion of health facilities with no stock outs of second line anti-malarial drugs during the last one month Malaria Case Fatality Ratio

Proportion of severe malaria cases treated according to national policy at health facilities

Social Behaviour Change Communication

Proportion of people aware of correct treatment for malaria Proportion of caregivers who know that children under five with fever should be seen by a health provider within 24 hours of onset of fever

Proportion of children with fever who received appropriate treatment within 24 hours Proportion of people aware of malaria prevention measures;

Proportion of children under five years old who slept under an ITN the previous night

Proportion of pregnant women who slept under an ITN the previous night Proportion of people who slept under an ITN the previous night

C. OUTPUT INDICATORS

Insecticide Treated Nets

Number of ITNs distributed, by target group Number of ITNs sold in the commercial sector (includes full cost and subsidized)

Number of distributors trained in ITN distribution

Indoor Residual Spraying Number of targeted structures sprayed

Number of districts covered by IRS Proportion of targeted houses sprayed with a residual insecticide in the last 12 months

Number of personnel trained in IRS

Malaria in Pregnancy Number of pregnant women receiving IPTp (1,2 or 3);

Number of SP doses distributed to ANC clinics;

Number of ANC health workers trained in IPTp

Diagnosis Number of RDTs distributed to health facilities;

Number of microscopes distributed to health facilities;

Number of health workers trained on RDTs (disaggregated by public health facility and non-public);

Number of health workers trained on microscopy (disaggregated by public health facility and non-public);

Cumulative number of drug distributors re-oriented on the use of RDTs;

Number of suspected malaria cases that have laboratory diagnosis Number of health workers in public health facilities trained in malaria RDTs in the targeted districts

Number of health workers in public health facilities trained in microscopy in the targeted districts

Case Management

Number of uncomplicated malaria cases reported from private not-for-profit facilities;

Number of under 5 children that received appropriate antimalarial treatment by a VHT member

Number of ACTs distributed to health facilities;

Number of uncomplicated malaria cases treated within public and PNFP health facilities;

Number of health workers trained in malaria case management (disaggregated by public health facility and non-public);

Cumulative number of drug distributors re-oriented on the use of ACTs;

Number of children under 5 years of age treated with ACT within 24 hrs at community level using the home based management of malaria fever strategy

Proportion of CMDs without stock out of ACTs for more than 1 week

Number of uncomplicated malaria cases reported from private for-profit facilities Number of health staff trained in Integrated Community Case Management (ICCM)

Epidemic Preparedness and Response Proportion of epidemic prone districts with rapid response team trained in malaria EPR

Proportion of HF in epidemic prone districts with up-to-date normal channels

Social Behaviour Change Communication Proportion of primary schools with at least two teachers trained in malaria prevention and control

Monitoring, Evaluation and Research Number of quarterly malaria surveillance and performance reports produced from NMCP

D. PROCESSES INDICATORS

Insecticide Treated Nets

Number of ITNs procured for free distribution Number of ITN guideline books distributed

Number of ITNs distributed through routine keep up distribution (ANC services to pregnant women and EPI)

Indoor Residual Spraying Number of insecticide susceptibility tests conducted

Number of bioassays studies done Number of entomological studies done

Quantity of insecticide (sachets) procured

Diagnosis

Number of RDTs procured for public sector Number of microscopes procured for public sector

Case management Number of ACT treatments procured for public sector

Number of fever cases receiving subsidized branded ACTs through the private sector Proportion of targeted health facilities supervised

Social Behaviour Change Communication Number of CSO coordination meetings held

Monitoring, Evaluation and Research Proportion of malaria control indicators reported (including narrative) on time at national level and disseminated to all levels

Number of quarterly malaria surveillance and performance reports produced Number of partner reports submitted

Total number of studies conducted (segregated by intervention) Number of operational research studies coordinated by NMCP

Program Management

Proportion of targeted health facilities supervised Number of RBM meetings held

Number of partners attending RBM meetings Number of districts receiving at least 1 technical support supervisions per quarter (where the program will include zonal coordinators)

Proportion of budgeted funds received Proportion of received funds spent

4 DATA COLLECTION AND REPORTING

4.1 DATA COLLECTION METHODS

Data collection for M&E indicators will utilize both qualitative and quantitative methods and, as much as possible, employ standardized data collection tools and analysis techniques. Most data will be collected routinely and any survey based indicators will be collected at baseline, midterm and at the end of implementation of the strategic plan.

4.2 DATA SOURCES

The data needs of the NMCP are based on agreed performance indicators to facilitate monitoring, evaluation, reporting and decision making. Collecting, analysing, interpreting, and reporting on the strategic information from stakeholders, forms a crucial part of national M&E activities. Relevant information for monitoring and evaluating progress in national malaria control comes from many sources and stakeholders encompassing governmental, nongovernmental, private, and international agencies.

It is therefore important that data sources report as appropriate and in a timely manner to meet the reporting needs and for timely data utilization. Data sources include standard reports from the National Malaria Control Program and other government line ministries, routine reporting from national surveillance systems and partners, and periodic household surveys such as DHS, and facility surveys. An overview of these sources is illustrated in Figure 5.

4.2.1 ROUTINE DATA COLLECTION

a) Health Management Information

System

The HMIS serves as the primary clinical services monitoring system for the MoH. The program is working with MoH Resource Centre to ensure key malaria information reported and disaggregated by sex, age (less than five years and 5 years and above) by the HMIS include suspected malaria cases at OPD, number of suspected malaria cases tested by microscopy and RDT, number of confirmed malaria cases at OPD, number of inpatient malaria cases including in pregnant women, number of clinical and confirmed inpatient malaria cases, number of malaria deaths, antenatal attendance, IPT1 and IPT2. The HMIS also collects information on stock out of first and second line anti-malarial medicines and other health management indicators.

The transformation to DHIS 2, which is expected to improve timeliness and completeness of HMIS, will include more malaria data reported at district level by end of 2012. There are still opportunities to incorporate malaria data with other management data currently being developed with technical assistance from partners, especially in those supporting health systems strengthening (SURE).

Figure 5: Sources of data

The program is working towards strengthening regular data analysis and review at health facility, district and national levels, supporting the mechanism for data collection and reporting from private sector health care facilities, operationalizing the NMCP composite malaria database and assigning responsibilities for its routine and overall management, developing standard reporting templates for partners to facilitate the incorporation of partner data into the NMCP database and supporting quarterly review meetings of health workers with CMDs/VHTs at sub county level and timely analysis and dissemination of data at all levels.

b) IRS monitoring system

Data collected includes entomological monitoring for choice of effective insecticide, number of supervisors and spray operators trained, number, IEC/BCC activities and proportion of structures sprayed, population protected by IRS as well as net coverage and usage. Reports on IRS implementation are initiated by supervisors at community levels, aggregated at district level by malaria focal persons, and then completed the form of program activity reports. At the various levels of compilation, the reports are used to guide corrective action as well as enhancing efficiency, performance and quality. A national monitoring and evaluation toolkit for indoor residual spraying will be developed for implementers and supervisors at different levels in the country.

c) ITN monitoring system

In 2007, the NMCP in collaboration with PMI developed and implemented an Excel data warehouse for tracking ITNs imported into the country. This collaborative project developed a composite database tool with various sources of relevant LLIN information at national level. The database harmonized the reporting requirements of the National Malaria Prevention and Control Monitoring and Evaluation Plan, the NMCP, Ministry of Health, and also the Health Sector Strategic Plan 2005-2010 indicators. The ITN database is used to monitor ITN partner activities and coordinate prospective ITN distributions to fill coverage gaps in specific sub-counties. The information generated by this system will feed into the NMCP Composite Database, though it is currently maintained separately due to problems in the functionality of the composite database and the urgent need to track large volumes of GF Round 7 Phase 1 nets to ensure proper distribution and tracking.

Mass campaigns such as the recently concluded Global Fund Round 7 Phase 1 involve a lot of actors’ at all administrative levels in the country in order to reach the beneficiaries within every targeted household. Using predesigned reporting forms, all the actors have as part of their duties to fill out numbers of beneficiaries within their areas of jurisdiction and to aggregate this information for sub county and district level data. See Annex 2 for more details. It is from the initial household beneficiary registration and the final distribution data that the denominator and the numerator for the campaign indicators are generated respectively. Because all must be done to ensure the highest quality data management through a pragmatic training cascade involving all levels of the hierarchy, IEC/BCC, timely payments and close supervision.

d) Integrated Disease Surveillance (IDSR) Adopted in 2001, Integrated Disease Surveillance and Response in Uganda is the weekly epidemiological surveillance reporting system that reports on diseases of epidemic potential.

This system provides data on malaria cases and deaths on a weekly basis. The Resource Centre has expanded the IDSR to incorporate more data on malaria.

e) mTRAC (medicine Tracking) There have been intermittent stock outs of antimalarials in some health facilities, overstocking and/or expiries in other health facilities, with poor tracking of stocks overall. In addition, it hasn’t been possible to accurately quantify consumption of ACTs and RDTs. In order to overcome these shortfalls, a system utilizing mobile phones to submit real-time data (using Rapid SMS technology) has been introduced and is being scaled up. This system will monitor ACTs and RDTs and once operational and functioning, other tracer drugs will be added and later incorporated in the DHIS 2.

4.2.2 HEALTH SURVEYS

These surveys are mainly carried out by Uganda Bureau of Statistics (UBOS) with support from partners and provide useful measures of household-based coverage indicators for gauging impact of interventions.

a) Community Surveys

• Demographic and Health Survey: The last Uganda Demographic and Health Survey (DHS) was conducted in 2011 and included a malaria module with standardized questions on coverage of key interventions.

• Malaria Indicator Survey: The first Malaria Indicator Survey in Uganda was done in 2009 and will be conducted again in 2013.

• ACTwatch Study: Another such household surveys was the ACTwatch Study conducted in 2008 and another 2011 aimed to generate evidence for policy makers on methods to increase availability and decrease the consumer price of quality assured ACTs in both private and public sectors. The ACTwatch household survey aims to collect information on health seeking behaviour and appropriate treatment (ACTs) and will be conducted in 2012.

• Small Scale Studies: Smaller scale household surveys are conducted periodically when there is a specific question requiring an answer. For example, in 2010/11, surveys on ITN coverage and usage after a universal coverage campaign in western Uganda (Malaria Consortium); an anaemia and parasitaemia survey in northern Uganda comparing districts that have received IRS to a district that had not (PMI); and a survey assessing the impact of home visits to increase utilization of nets in eastern Uganda (NetWorks).

b) Health Facility Surveys

• Uganda Service Provision Assessment: In 2007, the Uganda Service Provision Assessment

(SPA) Health Facility Survey was undertaken. This assessment is a nationwide facility-based survey designed to collect information on the availability and quality of reproductive and child health care, infectious disease (malaria, TB and HIV/AIDS) services provided to men, women and children in public, private and not for- profit health facilities throughout the country.

• AMFm Monitoring: through the Global Fund, monitoring of availability and pricing of subsidized ACTs will be done countrywide.

4.2.3 OTHER COMPLIMENTARY METHODS

a) Therapeutic efficacy testing

Over the past decade, studies have been conducted to assess the efficacy of various antimalarial drugs in specific populations, particularly among young children. The results from these studies have been used to assess existing and changing policies for antimalarial drug use across the region, including the changes in treatment policy to artemisinin-based combination therapy for acute malaria illness. These studies are conducted every two years to ensure resistance to antimalarials is identified in a timely manner. Increasing resistance to ACTs internationally makes this monitoring critical to malaria control in the country.

b) Pharmacovigilance

The pharmacovigilance system in Uganda is not well developed. Based on the WHO model, the National Drug Authority of Uganda (NDA) has designed a generic form to collect passive reporting data on all medicines. However, the reporting of adverse drug reactions (ADR) as part of the pharmacovigilance system in Uganda is not fully functional, nor utilized. Partners have supported the development and limited roll out of the system, providing training, forms and equipment, but reporting has been limited and follow up remains difficult as reports often come late and cases, thus, difficult to identify.

c) Health facility based surveillance via sentinel sites and DSS

• Sentinel sites: Sentinel sites were first established by the Uganda Malaria Surveillance Project

(UMSP) and the MOH in 2001 to determine the efficacy and safety of antimalarial drugs in epidemiologically different sites. Currently, the sites provide data on malaria cases presenting at the health facility and data on case management practices. There are six inpatient and six outpatient sites, located in Kabale, Kanungu, Tororo, Mubende, Apac, and Jinja. This data is reported through the routine HMIS, but also through a monthly report, disseminated to malaria stakeholders and an open-access website (www.umsp.muucsf.org).

• Demographic Surveillance Site: The Demographic Surveillance Site (DSS) is operated by Makerere University in Mayuge and Iganga districts. It monitors a defined population measuring births, deaths, and the most common causes of child mortality including malaria attributable deaths.

d) Activity Monitoring Systems/Activity reports

• District Level: Routine implementation reports are compiled to understand progress of district-level implementation of selected interventions. Examples of such reports include routine and activity-specific supervision, distribution of ITNs, SBCC activities, etc.

• National Level: At national level, compilation of activity/campaign reports by the program and the stakeholders are coordinated by the team leader/focal person responsible, sent to the relevant M&E unit for further analysis and synthesis of level of achievement of relevant indicators and compilation of performance reports. A number of reports (sometimes with unique formats) are required periodically from different national and international centres including the Focal Coordination Office for Global Fund, the WHO Country Office, PMI and others along the reporting hierarchy.

e) Insecticide Susceptibility studies

Uganda has had a long history of insecticide use for both public health and agriculture. As such, insecticide resistance testing is critical to ensure that effective chemicals are used for malaria vector control. Resistance testing has been conducted every two years (starting in 2009 and again in 2011) across the country testing all WHOPES approved chemicals for IRS and ITNs.

These studies have led to changes in the insecticide used for IRS and will inform the development of a rational vector control strategy for the country.

f) Other studies

From time to time, specific studies to answer internationally significant questions in malaria control and country-specific issues, operational research and studies on novel malaria control and treatment products and methodologies are carried out. These studies assist in improving current malaria control interventions and provide opportunities to implement new ones.

4.3 TIMEFRAME FOR M&E REVIEW

The frequency of measurement will depend on the place of the indicators within the M&E conceptual framework, taking into account a reasonable timeframe…