J.6_Performance-Based_Financing.pdf

Liberia PBF Operational Manual – Version 2

Republic of Liberia

Ministry of Health

Performance-Based Financing

Operational Manual

October 2016

Liberia PBF Operational Manual – Version 2 i

Contents List of Tables .......................................................................................................................................... iii

List of Figures ......................................................................................................................................... iii

Acronyms ............................................................................................................................................... iv

Preface ................................................................................................................................................... vi

Acknowledgements .............................................................................................................................. viii

I. Background

I.1. Country profile

II. Aim, Concept, and Definitions

II.1. Definitions

II.2. Objectives of Performance-Based Financing in Liberia

III. The PBF Institutional and Implementation Arrangements

Commonly Used Terminologies

III.1. Regulator

III.2. PBF Fund Holder

III.3. Purchaser

III.4. Implementers

III.5. Health Service Provider

III.6. Performance Verification at County Level

III.6.1. Brief on Data Collection and Entry in DHIS System

III.6.2. Verification of Service Delivery Indicators (Health Facilities)

III.6.2.1. Verification of Quantity Indicators

III.6.2.2. Verification of Quality Indicators

III.6.3. Validation of Health Facilities Invoices (County level)

III.6.4. Validation at the Community Level (County Level)

III.7. Counter-Verification by Central MOH

III.7.1. Counter-Verification of Administrative Indicators (Implementers’ Level)

III.7.2. Counter-Verification of Data at Health Facilities

III.8. PBF Oversight and Coordination at Central Level

III.9. PBF Oversight at County Level

IV. Tasks of Key Stakeholders

IV.1. Peripheral Level

Liberia PBF Operational Manual – Version 2 ii

IV.2. Intermediate Level – County Level

IV.2.1. District Health Team

IV. 2.2. Implementers (CHT and/or NGO)

IV.2.3. PBF County Steering Committee

IV.3. Central-Level MOH Key Players

IV.3.1. PBF Governance

IV.3.2. PBF Unit

IV.3.3. PBF Technical Committee

IV.3.4. External Aid Unit

IV.3.5. Office of Financial Management

IV.3.6. Procurement Unit

IV.3.7. HMER

V. Processes and Tools for Performance-Based Financing Implementation

V.1. Management of Performance-Based Financing

V.1.1.The Performance Contracts/Agreements

V.1.2. Development of Simple Business Plans for Performance Improvement

V.1.3. Performance Reports

V.2. Accounting and Financial Management

V.3. Information System

V.4. Performance Indicators

V.5. Setting Performance Targets

V.6. Bonus Computation

VI. Financing of PBF Scheme

VI.1. Process for Bonus Payment

VI.1.1. Bonus Payment for Health Facilities/Clinics

VI.1.2. Bonus Payment to Health Facility Staff

VI.1.3. Bonus Payment for Implementers

VII. Monitoring and Evaluation of PBF Implementation

VII.1. Levels and Actors of the Monitoring and Evaluation System

VII.1.1. First Level: Service Delivery - Recording and Verification of Data

VII.1.2. Second Level: County Level - Registration, Archiving, and Communication

VII.1.3. Third Level: Counter-Verification, Strengthening HIS, and Payment Approval

VII.2. Strengthening Capacity and Providing Support to PBF Actors

Liberia PBF Operational Manual – Version 2 iii

VIII. Conflict Resolution

IX. Annexes

Annex 1 – MOH PBF Indicators

Annex 2a – Sample Quarterly PBF Invoice for Health Facilities

Annex 2b - Data Counter-Verification Sheet

Annex 2c – Bi-Annual PBF Invoice for Implementers

Annex 3a – Sample Performance Agreement between MOH and Implementer

Annex 3b - Performance Agreement between Implementer and Health Facility

Annex 4 - Simple Business Plan for Health Facilities - MOH Liberia

Annex 5 – Reporting Tool for Health Facility (use of earned bonus)

Annex 6a – Summary Verification of Beneficiary’s Existence and Satisfaction

Annex 6b – Individual Questionnaire for Beneficiary’s Satisfaction

List of Tables

Table 1: Timeline of verification activities

Table 2: Calendar for Semiannual PBF Counter-Verification and Bonus Payment for IPs

Table 3: The Simple Performance Business Plan Format

Table 4: Summary Report of the Bonus Allocation

Table 5: Health Facility Staff – Bonus Distribution Worksheet

List of Figures Figure 1: PBF Institutional and Implementation Arrangements

Figure 2: Verification of Service Delivery Indicators (Health Facility)

Figure 3: Validation of Service Delivery Indicators (County Steering Committee)

Figure 4: Semiannual Counter-Verification by Central Level

Figure 5: Bonus Allocation MOH PBF Scheme

Liberia PBF Operational Manual – Version 2 iv

Acronyms ACT Artemisinin-based combination therapy

ANC Antenatal care

BPHS Basic Package of Health Services

CBO Community-based organization

CHB County health board

CHDC Community health development committee

CHO County health officer

CHT County health team

CM Certified midwife

CSO Civil society organization

DHIS District Health Information System

DHO District health officer

DHS Demographic and Health Survey

DHT District health team

EPHS Essential Package of Health Services

EPI Extended program for immunization

FARA Fixed Amount Reimbursement Agreement

FY Fiscal year

HF Health facility

HIS Health Information System

HMER Health Management Information System, M&E, and Research unit

IP Implementing partner

LMIS Logistics Management Information System

MCH Maternal and child health

M&E Monitoring and evaluation

MOH Ministry of Health

Liberia PBF Operational Manual – Version 2 v

NDS National Drug Service

NGO Nongovernmental organization

NHPP National Health Policy and Plan

OFM Office of Financial Management

OIC Officer in charge

ORS Oral rehydration salts

PBC Performance-based contracting

PBF Performance-based financing

PMTCT Prevention of mother-to-child transmission

QMU Quality Management Unit

RBHS Rebuilding Basic Health System project

RDT Rapid diagnostic test

TB Tuberculosis

USAID United States Agency for International Development

WB World Bank

Liberia PBF Operational Manual – Version 2 vi

Preface Performance-based financing (PBF) is an implementing strategy of the National Health

Policy and Plan (NHPP). The NHPP’s overarching goal is to equitably improve the health and social welfare status of Liberians. The Ministry of Health (MOH) aims to strengthen accountability and efficiency of service delivery through the separation of functions by contracting health program implementers through PBF, while the Central MOH focuses on the regulatory and resource-mobilization roles in the delivery of the Essential Package of

Health Services. Additionally, the MOH aims to shift focus from input to results and to motivate health workers and other stakeholders in an effort to improve their performance.

The MOH will be responsible for leading PBF in the health sector and will establish adequate regulatory capacity in the fields of legislation, standards-setting, operational guidance, and monitoring and evaluating performance. The PBF Operational Manual details: a) the relevant institutional and implementation arrangements; b) the roles and responsibilities of different stakeholders; and c) the implementation instruments, such as the Performance Improvement

Business Plan Guide, the methodology for setting performance targets, the methodology for verifying and reporting performance, and the tools for monitoring and evaluation (M&E). It ensures the key PBF functions are implemented by avoiding, to the extent possible, conflicts of interests in its implementation and validation of performance. PBF functions include: the regulation, fund-holding, and purchasing of the EPHS; provision of services; and independent verification and M&E of PBF implementation.

The PBF Operational Manual is built on global experiences and best practices of ongoing

PBF in the health sector in Liberia. Successful practices extend to areas such as: the utilization of the dashboard for PBF M&E, the elaboration of the performance business plan by the facilities, the performance agreements at all levels, and the joint verification of the facility performance by the county health teams (CHTs) and the implementer organizations.

The process of designing the PBF Operational Manual, Version 1, included a joint World

Bank (WB) and Rebuilding Basic Health Services (RBHS) assessment conducted in May

2011, a stakeholders’ workshop for validation of the performance-based contracting (PBC) assessment, selection of the preferred options for the operational manual, and a training workshop in June 2011 with the county monitoring officers and technical staff of the PBC implementing partners. The consultation process also included two debriefings about the proposed PBC operational arrangements with MOH decision-makers (minister, deputy and assistant ministers, and the directors and coordinators of national departments and programs).

The process leading to the current update of the operational manual (to Version 2) includes a consultative process with the different programs within the central MOH, senior MOH staff, and counties and partners supporting PBF innovation (USAID and WB).

This manual has been developed to reflect the implementation arrangements of the pilot PBF mechanisms ongoing in the health sector in Bong, Lofa, and Nimba counties. The PBF being rolled out in six hospitals, starting at the end of 2016 and early 2017, and in Sinoe, Rivercess, Liberia PBF Operational Manual – Version 2 vii and Gbarpolu counties with WB support, are being finalized and will be harmonized so far as possible with the ongoing PBF mechanism. They will both be incorporated into this manual, which will be continually updated to reflect developments in all three PBF approaches, with important lessons learned from these three approaches. The PBF operational plan will require some imperative adjustments during the process of scaling up of PBF at the national level.

The adjustments will be based on evidence of the best practices and constraints of PBF implementation in Liberia. Operations research interventions, in partnership with research institutions and development partners, will constitute an important component of the scheme.

Finally, the PBF manual will provide guidance for proper implementation of performance agreements leading to improved health for Liberians.

Honorable Dr. Bernice Dahn Minister of Health, Liberia

Liberia PBF Operational Manual – Version 2 viii

Acknowledgements Performance-based financing (PBF) is among the strategies being used to implement the

National Health Policy and Plan in Liberia. It aims to strengthen linkages between financial resources and health outcomes. The PBF operational manual, which will clarify and simplify

PBF implementation processes for all stakeholders at different levels, is the result of multiple consultations with and contributions from counties, Ministry of Health programs, and local and international partners.

I wish to acknowledge the valued contributions of the Bong, Nimba, and Lofa county health teams, the PBF steering committees in those counties, the Nimba superintendent’s office, National Malaria Control Program, National AIDS Control Program, Health Information

System Unit, Quality Management Unit, County Health Services Unit, Expanded Program on

Immunization, Family Health Unit, Community Health Services Unit, Supply Chain

Management Unit, Global Fund Project Coordination Unit, Fixed Amount Reimbursement

Agreement (FARA) office, Pool Fund office, and World Bank Project Implementation Unit.

The ministry also recognizes health partners for their technical inputs, particularly the World

Health Organization, United Nations Population Fund, World Bank, International Rescue

Committee, Africare, the US Agency for International Development (USAID) Liberia

Collaborative Support (CSH) for Health Program, and USAID Liberia office.

However, on behalf of all who contributed, I wish to especially acknowledge the PBF manager, Vera Mussah, for her managerial and technical leadership in this work; other PBF unit team members, including Dominic Togba, Ishmael N. Belleh, and Catherine Trawally;

Mike Mulbah, monitoring and evaluation officer; and Louise Mapleh, FARA manager, for their valuable inputs and tireless work in completing the PBF manual; and lastly Dr. Floride

Niyuhire from USAID CSH for her technical assistance.

Dr. Francis N. Kateh, Deputy Minister of Health Services and Chief Medical Officer

Liberia PBF Operational Manual – Version 2 1

I. Background

I.1. Country profile

Liberia has 15 administrative counties. One-third of all Liberians live in Monrovia, the capital. Nationally, 47 percent of the entire population lives in urban areas. The county health teams (CHTs) are the operational arm of the Ministry of Health (MOH). Under the direction of the Central MOH, CHTs manage all ministry-owned facilities, ministry-employed human resources, and ministry-provided material resources in their counties. A county health board

(CHB), chaired by the county superintendent, advises the CHTs, assists with resource-mobilization and coordination, and monitors the implementation of the County Health Plan.

In some districts, generally with a secondary care facility, a district health officer (DHO) coordinates district planning and supervision. Not all counties have fully developed health districts. In these cases, the DHO is based at the county level and coordinates planning and supervision in assigned facilities. Other counties have fully developed health districts, with district health teams (DHTs); this is the case in Bong, Nimba, and Lofa.

The 2013 Demographic and Health Survey (DHS) reported a maternal mortality ratio of

1,072 deaths per 100,000 live births, an infant mortality rate of 54 deaths per 1,000, and an under-five mortality rate of 94 deaths per 1,000. The total fertility rate is 4.7 and the contraceptive prevalence rate is 11%. The DHS also reported skilled attendance at birth at only 61.1% and deliveries taking place in a health facility (HF) at only 55.8%. The full vaccination coverage rate is 54.8%. Thirty-two percent of children under-five have chronic malnutrition. Malaria, acute respiratory infections, diarrheal diseases, and malnutrition remain the main causes of under-five deaths. Malaria remains the leading cause of morbidity and mortality in Liberia. The 2013 DHS reported a nationwide HIV prevalence of 1.9%;

urban residents have a higher prevalence at 2.6% compared to rural residents at 0.8%.

Epidemiological mapping shows a wide distribution of onchocerciasis, lymphatic filariasis, soil transmitted helminthes, schistosomiasis, and leprosy in all counties.

From 2007 to 2011, the MOH implemented the Basic Package of Health Services (BPHS), a set of high-impact interventions that prioritized the services thought to be most critically needed to improve the health status of Liberians, including: 1) maternal and newborn health;

2) child health; 3) reproductive and adolescent health; 4) communicable disease control; 5) mental health; and 6) emergency care. In July 2011, the BPHS was expanded to the Essential

Package of Health Services (EPHS) (document available at www.moh.gov.lr/doc/Final%20EPHS.pdf). This expanded package includes all elements of the basic package as well as non-communicable and neglected tropical diseases, and provisions for strengthening the referral system from the community to the tertiary level of health services. In 2013, the accreditation and clinical assessment of 396 public HFs was conducted. While 73% of facilities met the health system accreditation criteria, the average score on clinical assessment, which assessed the quality of care across multiple clinical areas, Liberia PBF Operational Manual – Version 2 2 was 38%; additionally, only 14% of assessed facilities were found to be implementing the full EPHS package.

Additionally, human resources for health remain a challenge, as highlighted in the recent

2015 Liberia Health System Assessment report. Low and unstandardized salary structures contribute to high attrition rates and demotivation of the health workforce. As of February

2015, 41% of the 10,052 health workforce was not on government payroll. Many health workers, particularly staff in rural communities, lack housing and accommodation.

Deployment of health personnel to rural areas has been difficult because the salary is fixed to the facility instead of the staff, and there are inadequate incentives to retain skilled providers in remote areas. Thus, under the 2015 investment plan, overhauling human resources for health with a health workforce that is fit for purpose and equitably distributed and incentivized is a priority of the Liberian Government and the MOH.

The Liberia National Health Accounts reported total health expenditure of US$240.5 million in Fiscal Year (FY) 11/12 and $364.9 million in FY13/14; the sharp increase was in part due to response to the Ebola epidemic. Regardless, out-of-pocket expenditures are consistently high and represented 51% of total health expenditure in FY11/12 and 35% in FY13/14.

Additionally, there is a high donor dependency in the health sector. On average donor contributions have accounted for nearly 50% of total health expenditure since FY07/08 and accounted for 80% of the total resource envelope of health in FY15/16. On the other hand, the total government health expenditure represented 11.5% for FY13/14 and 12.4% for

FY14/15.

The National Drug Service (NDS) serves the Department of Health Services and manages the central warehouse and the regional depots. NDS is responsible for the procurement, distribution, and management of medicines for the public sector and nonprofit institutions.

Drug requisition forms from counties are compiled and sent to the NDS for procurement.

Funding for the procurement of drugs is fragmented: the Government of Liberia pays for drugs at facilities which it is exclusively supporting; bilateral donors pay for drugs at facilities supported through nongovernmental organizations (NGOs); and vertical program donors pay for drugs that are provided free to all public facilities. Constant stock-out of commodities at point of service delivery led to ongoing rethinking of ways to optimize supply of commodities; this is yet to be finalized.

The MOH has undertaken efforts to strengthen collection of health-related information.

Health service delivery information is already captured in the Health Information Systems

(HIS); supply chain information is captured in the Logistics Management Information System

(LMIS); and other health information systems are under development. The Health

Management Information System, Monitoring and Evaluation, and Research (HMER) unit has taken the lead in developing a national HIS strategy that will address its interoperability.

The ultimate goal is to channel all health information through HIS, including that pertinent to governance, financing, and human resources for health.

Liberia PBF Operational Manual – Version 2 3

II. Aim, Concept, and Definitions

The National Health Plan and Policy (NHPP) aims to equitably improve the health and social welfare status of Liberians. In an effort to achieve this goal, the MOH established a policy for contracting the implementers of health programs and is committed to leading performance-based contracting/financing (PBC/PBF) in the health sector and establishing the adequate regulatory capacity in the fields of legislation, standards-setting, operational guidance, and monitoring and evaluation (M&E) of the performance.

Contracted implementers (CHT and/or NGO) will be responsible for implementing the

EPHS. The MOH intends to strengthen accountability and efficiency of invested resources through the separation of functions. In this paradigm, the central MOH will focus on the regulatory and resource-mobilization roles in the delivery of quality EPHS.

II.1. Definitions

PBF refers to the transfer of incentive (mainly payment based on fee-for-service or other monetary payment) from a funder or other supporter to a recipient, conditional on the recipient taking measurable action or achieving a predetermined performance target.

Recipients may be institutions and/or individuals (HFs, clinics, and/or health care providers).

On the other hand, PBC uses a supply-side PBF approach involving the development of a contract or other formal agreement.

A results-based financing glossary developed by the World Bank (WB) notes that: “PBC refers to contracts between a financing agent and an NGO, with payment depending on achievement of a performance measure that includes coverage targets and quality norms for a set of services. The contrast with PBF is that the latter concentrates on agreements with providers. Of course, the NGO operating under a contract may also be the provider or may in turn contract with providers, so the distinction between PBF and PBC is not strict; and PBC can be used more generally to refer to any contract where payment depends on a specific definition of performance.”

The Liberian national scheme will be referred to as PBF. This will involve three processes: 1)

Contractual arrangements between the MOH and CHTs (contracting-in); 2) Contractual arrangements between MOH and NGOs/civil society organizations (CSO) (management contracting); and 3) Performance agreements between the implementing partner (IP) and

HFs. This manual addresses procedures for PBF implementation only in selected health centers and clinics at the primary level that are financially supported through the Fixed

Agreement Reimbursement Amount (FARA). However, there are plans underway to introduce a modified model of the PBF scheme at primary HFs in selected counties with funds from the WB Ebola Early Recovery Plan. The manual is adaptable to other models that would address PBF best practices and promote the national health agenda, including the

Health Investment Plan and the EPHS.

Liberia PBF Operational Manual – Version 2 4

II.2. Objectives of Performance-Based Financing in Liberia

Existing policies, including the National Health Plan and Policy (2012-2021) and the

Liberian Investment Plan for Building a Resilient Health System (2015-2021), allow for the provision of performance-based services by contracting health services. For instance, the

MOH contracting policy states that the MOH will lead PBC in the health sector and will establish adequate regulatory capacity in standards setting and operational guidance.

Standardized contracts and reporting and monitoring tools will be developed. A mixture of approaches will be tried. This includes: a) contracting in, when one level of government contracts with another; b) contracting out, when a partner is contracted with complete authority over all resources (human, material, and financial) to provide health services; and c) management contracting, when a partner is contracted to provide management services alone over government resources. Specific objectives of PBF are to:

accelerate gains in the implementation of EPHS and the national health investment plan through innovative health financing scheme;

promote accountability and evidence-based decision-making processes for the improvement of essential health services at all levels, especially in underserved and targeted populations;

improve local stewardship and ownership of health systems management in a decentralized manner; and improve the delivery of quality of health services by enhancing staff motivation through the provision of healthy competitive motivational packages.

III. The PBF Institutional and Implementation Arrangements

Commonly Used Terminologies

The regulator is responsible for the development policy, norms and standards – The

MOH is usually the regulator

The fund holder is responsible for disbursement of funds to service providers – The fund holder can be a donor, the public sector (government), or private sector (NGO, community-based organization [CBO]).

The purchaser is responsible for setting the health targets and monitors and verifies that results were achieved before making payment to the service provider – The purchaser can be can be a donor, the public sector (government), or private sector

(NGO, CBO).

The provider is responsible for provision of services – The provider can be the public sector (government) or private sector (NGO, CBO).

Liberia PBF Operational Manual – Version 2 5

The verifier (or controller) controls the level of performance – The verifier is an intermediary between the purchaser and provider and can be the public or private sector or include both (oversight committee).

The PBF scheme implemented at the primary level in Liberia can be broken down into five key functions: (i) regulation; (ii) fund holding/purchasing; (iii) implementing; (iv) health service provision; and (v) verification and counter-verification. Figure 1 represents the proposed institutional and implementation arrangements including these five key functions.

These are also described in subsequent text. Details of the roles and responsibilities of each stakeholder are explained in Section IV.

Figure 1: PBF Institutional and Implementation Arrangements

III.1. Regulator

The central MOH is the regulator in the PBF scheme and is responsible for the development policy, norms, and standards of care, along with ensuring they are enforced. The regulator ensures that the PBF scheme is aligned with the MOH guiding policy and procedures.

The PBF scheme is implemented through the MOH system, and the different departments play distinct roles. Specifically, the Department of Administration is responsible for the financial aspects, the Department of Health Services is responsible for the technical programmatic aspects, and the Department of Research and Planning is responsible for M&E and for verifying performance achievement.

Liberia PBF Operational Manual – Version 2 6

III.2. PBF Fund Holder

The fund holder is responsible for disbursement of funds to implementers according to the contractual agreement and disbursement schedule. This function is fulfilled by the MOH

Office of Financial Management (OFM) situated in the Department of Administration. While donor-supported programs within the MOH provide programmatic oversight to a funded program, with regards to financial management, they use the MOH system. This is accurate for programs such as FARA or the WB-supported program – Program Implementation Unit.

OFM disburses funds after performance verification and review of invoices by MOH, specifically HMER and PBF unit.

Therefore, The MOH OFM is the current PBF fund holder. Disbursements are done according to formal agreements, which strengthen the accountability of different stakeholders. Both HFs and county health teams will be held accountable for the attainment of targets, with the MOH providing the stewardship to ensure that all levels work collectively to improve the health system.

III.3. Purchaser

The purchaser is responsible for setting the health targets and monitors and verifies that results were achieved before making payment to the service provider. In the Liberian scheme, this function is fulfilled by the central MOH; different departments/units share this function.

HMER and PBF units are involved in setting targets, monitoring performance and verifying achievement, whereas the invoices are paid by the OFM.

Thus, the functions of fund holder and purchaser overlap in this scheme.

III.4. Implementers

The IPs (CHTs or NGOs) are responsible for ensuring availability of the highest possible quality of health services for the population in the catchment area as per the terms and conditions of the contract with MOH. By this, the implementer shall provide essential inputs as well as technical and managerial oversight required for creating enabling environment and making health service more responsive to prevailing public health demands.

The MOH shall meanwhile assess county capacity and readiness for contracting in, overtime, and shall determine, pursuant to such readiness assessment findings, whether to implement a capacity-improvement plan or to contract in a county for management and implementation of a performance-based contract. Already, a prototype hybrid management system of PBF implementation is evidenced in Bong.

The implementer is responsible for building the capacity of HFs, supplying key inputs, and providing trainings, supportive supervision, data management and reporting, outreach services, and community health services.

Liberia PBF Operational Manual – Version 2 7

The decision as to whether a county or NGO will be contracted as implementer rests with the

MOH. The capacity of a given county to manage significant funds and ensure effective coverage of all health services in catchment areas is among factors to consider.

Generally, there are two contracting modalities under this PBF scheme: management contracting (otherwise referred to as contracting-out) and contracting-in.

1. Contracting-out/Management Contracting: The MOH or fund holder contracts the managerial service of NGOs to manage and ensure health service delivery in selected

HFs. In this case, the NGO submits a budget which includes all necessary activities within a given period, including the amount for overhead. The contracted NGOs receive 90% of agreed-upon funds to initiate delivery of health services. NGOs ensure that the HFs in their catchment areas meet quarterly performance targets. NGOs and county health teams verify services provided, while the Central MOH is responsible for the counter-verification of implementers’ indicators. An additional 10% bonus is allocated based on performance of both the implementer and the HFs in the catchment area.

2. CHT (“contracting in”): Similarly, the contracted county will receive 90% of the funds to initiate delivery of health services. The county ensures HFs in its catchment areas meet quarterly performance targets. The CHT can sub-contract management of some HFs to an NGO implementer. The CHT and relevant partners verify services provided while the Central MOH is responsible for the counter-verification of implementers’ indicators. An additional 10% bonus is allocated based on performance of both the implementer and the HFs in the catchment area.

To strengthen the supervisory and supporting functions of the implementer, the selected PBF indicators will be based on a mix of supervisory/management performance and clinical results. Current performance indicators for implementers along with their assigned weight can be found in Annex 1.

III.5. Health Service Provider

The health clinics and centers are the PBF health services providers. They are responsible for delivering quality health services available as per the EPHS and for achieving performance targets. The PBF health services indicators are linked to the EPHS.

To ensure the quality of health services delivered, the PBF selected indicators will be based on a mix of administrative and management areas, outputs and outcomes, and the clinical and quality criteria. The PBF indicators will be selected based on rigorous consultation with relevant program managers and directors. They will be weighted and validated with all stakeholders, CHTs, and implementers and then enrolled into the MOH HIS. Current performance indicators and assigned weights for HFs and clinics can be found in Annex 1.

Liberia PBF Operational Manual – Version 2 8

The health HFs will produce the HIS monthly report by filling out the integrated report form and will highlight the PBF indicators for all stakeholders. Copies of the report will remain at the facility for verification and collection. At the end of the quarter, the HF will fill out an invoice which highlights the targets and achieved performance for each PBF indicator. The

DHT, under CHT supervision, is responsible for collecting the monthly report and the quarterly invoice filled out by each HF. This will remove the burden from the facility to transmit the report and standardize data collection.

III.6. Performance Verification at County Level

III.6.1. Brief on Data Collection and Entry in DHIS System

All service delivery data are generated at the facility level. The officer in charge (OIC) ensures the integrated report form is filled out promptly and correctly each month and signs off on. Additionally, at the end of the quarter, the HF will fill out the PBF invoice. The DHOs collect the integrated report forms and the quarterly PBF invoices from the HFs under their oversight and ensure the forms get to the county health office by the 5 th of each month. The data is then entered into the DHIS at the county level. Those two steps (collection and entering of data in DHIS) take about 2 weeks, and the data is then transmitted to the HIS by the 15 th of each month. The county HIS Unit takes an extra seven to 15 days to conduct the quality checks and resolves pending queries; hence, within 30-31 days following the end of the quarter, HIS data must have been verified, entered, and validated at the county level to inform county quarterly report submission. Meanwhile, MOH Central provides feedback on data based on outliers, report coverage, and obvious inconsistencies immediately after the th of the subsequent month, which is also the due date for CHT posting of HIS data to the server.

III.6.2. Verification of Service Delivery Indicators (Health Facilities)

IPs (NGOs or CHTs) use two components to verify PBF performance indicators:

Quantity indicators

Qualitative indicators

III.6.2.1. Verification of Quantity Indicators

The purpose of verification, or cross-checking, is to strengthen the quality of the data collected at HFs and captured in the District Health Information System (DHIS), and to confirm achieved performance for PBF indicators. Implementers (CHTs and NGOs) are responsible for coordinating logistical support and for conducting this quantitative verification each quarter.

For better coordination, efficient use of resources, and strengthening of HIS data, the verification of PBF indicators will collaborate with the clinical supervision and M&E tasks conducted each month. For instance, the CHT, which is in charge of clinical supervision, has monthly and quarterly scheduled supervisory activities. The county clinical supervisor, Liberia PBF Operational Manual – Version 2 9 county M&E officer, county data officer, and M&E officers from varied programs (extended program for immunization [EPI], malaria, HIV, family planning, etc.) contribute to this activity. The PBF verifier(s) and the team conducting the clinical supervision and M&E will train together in order to conduct the verification of PBF indicators, per contractual agreement.

With the above workforce, each county will be able to have approximately three teams of verifiers, with each team comprised of three members. Each team will be assigned to 7-10

HFs. Thus, the tasks shall be completed in three weeks. The verification team cross-checks the data collected through the DHIS with the source data at the facility (i.e. ledgers, registers), and conducts the verification of PBF indicators. Discrepancies identified during the verification exercise will be brought to the attention of the data manager at the county level, who will amend the records in the DHIS. Thus, at the end of this process, records sent to the Central MOH would have been verified and deemed accurate. The implementers verify

HF PBF service delivery indicators each quarter.

The verifier will have accessible a copy of the HIS report for the corresponding period, to which is annexed a report of “declared data” (Annex 2a), corresponding to services provided by the HF regarding PBF performance indicators. The verifiers will cross-check against the original data in registers and other documents held by the HF. This verification will yield two types of reports:

1. HIS verified data: consists of accurate quantitative data of services provided at the

HF, regardless of whether information on required identifiers or care received by the service beneficiary is complete or incomplete.

2. PBF indicators verified data: consists of quantitative data for services provided with regards to PBF indicators and will only include verified data that meets the conditions as specified in the performance agreement (i.e., completeness of all required information beneficiary’s name, age, address, diagnosis, treatment, date).

Certain types of discrepancies will be more severely sanctioned than others. For example, knowingly falsifying the data in order to increase the earned bonus will result in withholding of any bonus earned on the concerned indicator by the HF for that quarter. If this is a repeat offender situation, the concerned HF may be debarred from participating in the PBF scheme for a determined period of time.

The exclusion of repeat offenders from participating in the scheme will be proposed by the implementer, who will supply documentation of the offenses; the county PBF steering committee will review and approve the decision. The decision will be communicated to the

PBF Unit, which will learn from the experience and develop strategies, in consultation with counties, to mitigate similar situations in the future.

Finally, the HF will submit the quarterly invoice on site for the verifier (Annex 2a). The form has fields for declared data and for verified data. Upon completion, the OIC and the team that conducted the verification will sign the invoice. A copy of this invoice will remain at the HF

Liberia PBF Operational Manual – Version 2 10 for its records. The PBF implementer will transmit the PBF verified report to the PBF county steering committee for validation.

III.6.2.2. Verification of Quality Indicators

Allocation of a PBF bonus to HFs and counties based on score achieved on the regular annual accreditation and quality assessment failed to draw the desired attention to quality of care. In fact, quality assessments conducted in 2012 and 2013 show no improvement in quality of care, suggesting that this assessment alone was insufficient to motive the desired changes.

Moreover, the accreditation and clinical assessment were conducted annually; thus facilities were rewarded quarterly based on an annual quality score and have no real incentive to improve performance from one quarter to the next. Finally, the Ebola epidemic reinforced the need to further improve quality of services. Lessons from around the world have shown that quality of care can be successfully promoted through PBF mechanisms by introducing a PBF composite quality indicator. HFs earned a bonus for achieving the quarterly target set for the quality indicator.

Therefore, Liberia’s MOH decided to introduce a composite quality indicator. This indicator was developed with inputs from the recently created Quality Management Unit and will be aligned with the core clinical standards and the joint integrated supportive supervision tools.

The quality indicator will be applied to primary-level HFs implementing PBF mechanisms and will serve to track progress in quality improvement, provide feedback for quality improvement to HFs, and to determine whether HFs achieved agreed-upon quality targets in order to receive the PBF bonus payment.

Under each clinical technical area, there will be a set of quality standards to be checked on, along with the scoring criteria and the maximum points to be earned. The total score obtained on this quality assessment will be converted into a percentage, which will constitute the PBF composite quality index score.

DHTs will conduct monthly supportive supervision using MOH standardized checklist. The score obtained on qualitative assessment and the identified weaknesses will inform this supportive supervision and mentorship provided by DHTs.

A summary of the verification process at the facility level is illustrated in the following graph:

Liberia PBF Operational Manual – Version 2 11

Figure 2: Verification of Service Delivery Indicators (Health Facility)

III.6.3. Validation of Health Facilities Invoices (County level)

The validation of a HF’s results is the responsibility of the PBF county steering committee and will be conducted each quarter. It will take place once the IPs (CHTs and NGOs) verify the HF’s results – about six weeks after the end of the quarter. The documents submitted by the IPs to the PBF county steering committee include the quarterly report from implementers and invoices reflecting the achieved performance and earned bonuses. The latter include the score obtained on the client satisfaction survey as well by individual facilities.

The PBF county steering committee will review the procedures followed by implementers in their verification tasks, review achieved results by HFs, and check for any potential discrepancies. This validation of invoices is conducted at the county level and does not involve facility visits by the county PBF steering committee. The PBF county steering committee will then validate the invoice by co-signing. Afterwards, the implementers will submit the invoices for HF validated results to the External Aid Unit, with a copy to the PBF

Unit.

A summary of the validation process, including validation at the community level, is illustrated in the following graph:

Liberia PBF Operational Manual – Version 2 12

Figure 3: Validation of Service Delivery Indicators (County Steering Committee)

III.6.4. Validation at the Community Level (County Level)

Data validation at the community level aims to confirm the existence of beneficiaries randomly selected in registers at HFs and assess how satisfied the beneficiaries are with services received. Hence, validation at the community level assesses the subjective quality as perceived by clients; this allows collecting feedback from the beneficiary/patient that can be acted upon by service providers in order to improve the beneficiaries’ experience.

The findings of the validation exercise at the community level will be factored into calculating the final bonus earned by the HF. An indicator, “Score obtained by HF on perceived quality of services,” is included on the list of PBF indicators. Thus, this will bring renewed attention by health care workers to ensure patients have an improved experience during their visit at the HF. Detailed questions for the validation at the community level are included in the verification and validation tools (see Annex 6).

Establishing a System to Conduct the Community-based Data Validation

The validation of data at the community level is to be conducted quarterly. A third party, contracted by the county steering committee, conducts this activity. Community-based organizations (CBOs), especially those whose mission is to fight poverty or advance public health objectives (i.e. reproductive health, community health), can be sub-contracted as the third party to validate delivered services to beneficiaries. Only registered CBOs will be eligible. To limit conflict of interest, a CBO with representation on the PBF county steering

Liberia PBF Operational Manual – Version 2 13 committee shall not be eligible for these functions. Preferably, the local association will be assigned to the catchment area of a HF. Also, there shall be no commercial/business or personal ties between the HF and the local association conducting the validation in order to avoid conflict of interest.

CBOs can satisfactorily assess the subjective quality as perceived by beneficiaries, whereas assessment of objective quality of care would require more clinically qualified verifiers. The objective assessment of the quality of care is being strengthened elsewhere by introducing a step in the verification at facility level that aims to assess composite quality indicators. The results of the composite quality indicators will be converted into a quality index score.

The responsibility for validation of data at the community level is incumbent on the PBF county steering committee. The PBF and M&E units will assist the county steering committee in the initial stages and assist with capacity-building efforts toward fulfillment of this function by the PBF county steering committee. It needs to be emphasized that this validation may not be conducted directly by the PBF county steering committee. CBOs can, however, fulfill this mission under the guidance of the committee. Hence, this is a good opportunity to engage the communities in PBF implementation, and thereby favor sustainability of the scheme.

Determine Service Beneficiaries to be Validated

Preferably, two CBOs will be recruited per county. An average PBF county has about 40

HFs, thus each CBO will be assigned to approximately 20 HFs. Bigger counties may consider contracting more CBOs or assigning more resources to contracted CBOs so that they can cover more than 10 HFs. For each HF, two villages will be randomly selected and 10 beneficiaries of services will be selected in each, even though the goal is to interview only 5 beneficiaries. This was informed by the piloting of the dedicated tools and challenges encountered in locating the beneficiaries, mainly due to incomplete or inaccurate beneficiary identifiers, as recorded in HF registers. For each of the selected beneficiaries, the CBO will verify their existence as well as the subjective quality of service provided by the HF by administering a set of questions designed solely to capture the perceived quality of care. If the first 5 beneficiaries were easily found and interviewed, the surveyor need not interview all

10.

If any of the selected beneficiaries reside outside of the concerned county, it may not be economical to track them; thus, they will be replaced by another selection of beneficiaries. If the random selection yields an excessive number of beneficiaries who reside outside of the

HF catchment area and/or county, this could be an issue linked to the situation on the ground

(i.e., influx of refugees). The HF will not be penalized for such a situation. However, efforts should be invested in assuring it is not a fraudulent situation by the HF. The person conducting the validation must notify the PBF county steering committee of these incidences, and the steering committee will then decide if an investigation is warranted.

Conduct the Validation at Community Level

Liberia PBF Operational Manual – Version 2 14

The surveyor conducting the validation at the community level (i.e. local association) will be given a form with only necessary details to track the beneficiaries in the community (i.e. first and last name, county, district, locality, and/or name of chief of family) and key elements to be verified by the patient. Care will be taken to protect the privacy of individuals’ health information. Also, when the surveyors visit the household(s), the family will be informed of the purpose of the survey and will freely decide whether to respond.

Some of the information to be collected is aimed at ensuring the surveyors visited the households. For example, surveyors will need to request the age of the patient, note the

(approximate) date of the visit to the HF, length of stay (if patient was hospitalized in health center), gender, and, for pregnant women, the rank of pregnancy (i.e., 1 st , 2 nd , or 3 rd pregnancy). Field experience in testing the tools indicates that some beneficiaries don’t know their exact age or the exact time of the last visit. Thus, a note of caution for the commissioner of the survey in checking whether the surveyor truly visited the beneficiary – an estimate age that is +5 years recorded at the HF will be acceptable; likewise it is acceptable if beneficiaries give the approximate month and not the exact date they visited the HF.

Score Resulting from the Validation

The score resulting from validation at the community level will be recorded under the indicator: “perceived quality of services.” Details on the calculation of this score are included in the verification and validation tools; a sample summary sheet for scoring the community survey is attached in Annex 6a (extracted from verification and validation tools).

HFs are responsible for correctly recording patients visiting their institution. If beneficiaries could not be located due to incomplete identifiers recorded by the HF, the latter should receive feedback through the appropriate channel in order to improve record-keeping. Each identified case of a “ghost” patient will be given a score of zero; thus, this will affect the overall score and potential bonus to be earned by the facility. If this is a repeated offense, the concerned HF may be debarred from participating in the PBF scheme for a determined period of time.

Additionally, for traced patients, perceived satisfaction of services provided by a HF will be gauged and classified in three levels and given a score of either 0, 5, or 10 (i.e. waiting time at the HF will have three responses: too long, medium, short. They will respectively get a client satisfaction score of 0, 5, or 10. A sample for questions assessing client satisfaction is attached in Annex 6b (extracted from verification and validation tools).

A timeline of all verification activities is shown in the table below. It starts with end of the quarter and shows in sequence the timeline for verification and validation of results pertinent to HF performance.

Liberia PBF Operational Manual – Version 2 15

Table 1: Timeline of verification activities

III.7. Counter-Verification by Central MOH

The purpose of the counter-verification of the performance reports is to strengthen data quality by detecting errors and preventing over-reporting, especially given that there is a financial bonus linked to performance.

The counter-verification will be conducted on semiannually. To increase the level of integrity, avoid any conflict of interest, and ensure reliability of the results, to the extent possible, the counter-verification will be conducted by a party with no ties to the financial incentive resulting from the PBF scheme, or by a third party organization.