IPLS.pdf

I

Ethiopia: National Survey on the Integrated Pharmaceutical

Logistics System (IPLS)

September, 2018

Ethiopia: National Survey of the Integrated Pharmaceutical Logistics System

II

AIDSFree

The Strengthening High Impact Interventions for an AIDS-free Generation (AIDSFree) Project is a five-year cooperative agreement funded by the U.S. Agency for International Development under

Cooperative Agreement AID-OAA-14-000046. AIDSFree is implemented by JSI Research &

Training Institute, Inc. with partners Abt Associates Inc., Elizabeth Glaser Pediatric AIDS

Foundation, EnCompass LLC, IMA World Health, the International HIV/AIDS Alliance, Jhpiego

Corporation, and PATH. AIDSFree supports and advances implementation of the U.S. President’s

Emergency Plan for AIDS Relief by providing capacity development and technical support to

USAID missions, host-country governments, and HIV implementers at the local, regional, and national level.

III

Recommended Citation

Ethiopia Pharmaceutical Supply Agency (EPSA), 2019. Ethiopia, National Survey of the Integrated

Pharmaceutical Logistics System: AIDSFree, and Pharmaceutical Supply Agency (EPSA).

Contributors:

EPSA JSI SART

Admasu Teketel Marasi Mwencha Tseganeh Demissie

Siraj Adem Paul Dowling Michael Tirfe Fantaye Teka Sami Tewfik Yehenew Demelash

Tilahun Birhane

Welelaw Necho Andualem Mohammed

Abstract

Since 2009, the Pharmaceutical Supply Agency (EPSA) has implemented the Integrated

Pharmaceutical Logistics System (IPLS) to manage essential health commodities in the public sector. This report describes the findings from a survey on the status of IPLS implementation and its performance at health facilities within the public sector.

THE FEDERAL DEMOCRACTIC REPUBLIC OF ETHIOPIA

PHARMACEUTICAL SUPPLY AGENCY (EPSA)

Tel: +251.011.275.1770

Fax: +251.011.275.2555

P.O. Box 21904

Addis Ababa, Ethiopia

Email: pfsa@ethionet.et

JSI Research & Training Institute, Inc.

1616 Fort Myer Drive, 16th Floor

Arlington, VA 22209 USA

Phone: 703-528-7474

Fax: 703-528-7480

Web: aidsfree.usaid iv

11, CONTENT

TABLES ......................................................................................................................................................... vi

FIGURES ...................................................................................................................................................... vii

ACRONYMS ................................................................................................................................................. ix

Foreword ...................................................................................................................................................... x

Acknowledgement...................................................................................................................................... xi

INTRODUCTION

METHODOLOGY

Study Population

Sample size determination

Sample Selection Procedures

Survey tool

Data collection

Data management

Quality assurance

Ethical considerations

Limitations

Survey Findings and Discussion

Facility description

Logistics System Management Practices

Availability of SOPs and health post job aids

Logistics Data Recording Practices

Logistics Reporting Practices

Human resource and training

Professionals in pharmacy units

IPLS training

Storage conditions v

Storage conditions fulfillment

Cold chain equipment for vaccines storage

Transportation and distribution

Primary immediate sources of supply

Transportation and modes of delivery

EPSA delivery accompanied by pharmacy professional

Supervision

Supply chain performance

Stock availability

Emergency order

Wastage

Inventory turnover

EPSA Share in Product Supply

Order Fill Rate

Delivery Lead Time

Conclusions and Recommendations

Annex 1: Result Tables

Annex 2: Integrated Pharmaceutical Logistics System Survey in Ethiopia vi

TABLES

Table 1: Initial universe and original sample distribution by facility type

Table 2: Actual sample distribution by facility type and by region

Table 3: Bin card use and up datedness for priority indicator pharmaceuticals

Table 4: Bin card accuracy and near accuracy for priority indicator pharmaceuticals

Table 5: Bin card accuracy and near accuracy in 2018 and 2015 IPLS survey results

Table 6: vaccine ledger/register use and up datedness for Penta –Inj

Table 7: Vaccine ledger/register accuracy and near accuracy for Penta –Inj

Table 8: Types of logistics reports

Table 9: IFRR Schedule adherence by DUs

Table 10 near accuracy RRF and HPMRR data quality for priority indicator pharmaceuticals

Table 11: Percent of health facilities meeting individual storage criteria by facility type

Table 12: Primary immediate sources of supply

Table 13: Pharmaceuticals availability on the day of survey

Table 14: availability of Tracer items, time of visit Electronic) and paper base sites

Table 15: Average availability on the day of the survey in 2018 and 2015

Table 16: Percent of Health facilities stocked out at least once within the 6 months preceding the survey

Table 17: Frequency of stock outs within the 6 months preceding the survey

Table 18: Duration of stock out in the 6 months prior to the survey (days)

Table 19: Percent of facilities, understocked, optimally stocked and overstocked, by facility type

Table 20: Months of stock by HCMIS and paper-based facilities

Table 21: Perceived Order Fill Rate by a program, and facility level

Table 22: Tracer products and Abbreviated names

Table 23: Use of bin card by item and facility type

Table 24: Facilities where bin cards are updated by product and facility type

Table 25: Bin card use electronic Vs paper based

Table 26: Facilities with bin card accuracy by product and facility type

Table 27: Near bin card accuracy for all tracer products by facility types

Table 28: IFRR use in at least one and eighty percent of DUs by all facility type

Table 29: IFRR Schedule adherence by DUs by all facility types

Table 30: RRF completeness by program for all health facilities

Table 31: RRF data quality for all tracer products by all facility type

Table 32: Storage condition fulfillment comparison between 2018 and 2015 surveys

Table 33: Storage Condition Fulfillment for Health center

Table 34: Storage Condition Fulfillment for Health post vii

Table 35: Health facilities that had stock on hand of pharmaceuticals available on the day of the survey by health facility level (for 37 Pharmaceuticals)

Table 36: Health facilities with any stock out within the 6 months preceding the survey

Table 37: Health facilities with 1-2 times stock out within the 6 months preceding the survey

Table 38: Duration of stock out within the 6 months preceding the survey

Table 39: Month of stock by health facility type

Table 40: Month of stock by HCMIS and paper based health facilities

Table 41: Frequency of emergency order

Table 42: Wastage

Table 43: EPSA share from total supply by value

FIGURES

Figure 1: Sample distribution from a commodity management perspective

Figure 2: Distribution of sampled health facilities by level (%)

Figure 3: Health facilities by electronic (Dagu) and paper-based/manual

Figure 4: Health facilities with direct EPSA delivery modality

Figure 5: National level infrastructure availability

Figure 6: Facilities without access to electricity, internet, and mobile phone

Figure 7: IPLS SOP availability

Figure 8: Availability of health post job aids/flip books

Figure 9: Blank bin card availability

Figure 10: Blank bin card availability in 2018 and 2015 IPLS survey results

Figure 11: Bin card in use for at least one tracer product

Figure 12: Average use of bin cards for all tracer products assessed

Figure 13: Bin card use Dagu vs. paper based

Figure 14: Up-to-datedness of bin cards Dagu vs. paper based

Figure 15: Blank vaccine ledger/register availability

Figure 16: vaccine ledger/register use

Figure 17: IFRR use in at least one and eighty percent of DUs

Figure 18: IFRR Schedule posted in at least one DU

Figure 19: Facilities with a regular resupply schedule posted in 2018 and 2015 IPLS survey results

Figure 20: Blank RRF/HPMRR availability

Figure 21: Percent of facilities using RRF

Figure 22: RRF use HCMIS vs. paper-based facilities

Figure 23: HPMRR availability and use at health posts

Figure 24: Number of RRF reports submitted in the 3 review/reporting periods preceding the survey

Figure 25: RRF good reporting HCMIS vs. paper based viii

Figure 26: Good HPMRR reporting practice

Figure 27: Facilities having a recent complete RRF in the three reviews/reporting period preceding the survey

Figure 28: Facilities having a recent complete HPMRR in the three reviews/reporting period preceding the survey

Figure 29: Blank VRF availability

Figure 30: VRF use

Figure 31: Facilities having a recent complete VRF in the three reviews/reporting period preceding the survey

Figure 32: Near accuracy VRF data quality for the priority indicator pharmaceutical Penta –Inj .. 30

Figure 33: Number of health professionals assigned to the pharmacy units of health facilities ... 31

Figure 34: Pharmacy professionals assigned to the pharmacy units of health facilities

Figure 35: Facilities having non-pharmacy professionals in pharmacy units

Figure 36: Heath facilities having at least one IPLS trained pharmacy unit staff

Figure 37: Health facilities with at least 80% of pharmacy unit staff trained in IPLS

Figure 38: Hospitals and health centers having staff trained in IPLS with different modalities

Figure 39: Health posts having staff trained in IPLS with different modalities

Figure 40: Storage condition fulfillment

Figure 41: Percent of health facilities meeting at least 80% of common storage conditions, 2015 and 2018

Figure 42: Percent of health facilities with any refrigerator and with a functional refrigerator for vaccine management

Figure 43: Percentage of facilities with different source of energy for functional refrigerators

Figure 44: Availability of temperature recording device

Figure 45: Percent Functional refrigerators found to be within the desired temperature range

Figure 46: Percentage of facilities with temperature recording chart

Figure 47: Percentage of facilities with updated temperature recording chart

Figure 48: Pharmaceutical transportation modality

Figure 49: Facilities that picked up pharmaceutical by type

Figure 50: Means of transportation most commonly used by health facility type

Figure 51: EPSA delivery accompanied by pharmacy professional

Figure 52: Percent of health facilities that have received logistics related supervisory visits over time ... 45

Figure 53: Average availability of pharmaceuticals in 2018 and 2015

Figure 54: Percent of facilities placing emergency orders over previous six months, by type of facility ... 54

Figure 55: Means of placing an emergency order

Figure 56: Wastage

Figure 57: Inventory turnover at health facilities

Figure 58: EPSA share in total supply

Figure 59: Order Fill Rate for selected products, at health centers and hospitals

Figure 60: Percent on time delivery for the most recent period before the survey, hospitals and health centers ix

ACRONYMS

AMC Average Monthly Consumption

ART Antiretroviral Therapy

DU Dispensing Unit

EPSA Ethiopian Pharmaceutical Supply Agency

FEFO First-to-Expire, First Out

FMoH Federal Minister of Health

FP Family Planning

HCMIS Health Commodity Management Information System

HEW Health Extension Worker

HPMRR Health Post Monthly Report and Request

IFRR Internal Facility Report and Request

IPLS Integrated Pharmaceuticals Logistics System

LMIS Logistics Management Information System

MCH Maternal and Child Health

OPD Outpatient Department

ORS Oral Rehydration Salt

PFSA Pharmaceutical Fund and Supply Agency

PLMP Pharmaceutical Logistics Master Plan

RDF Revolving Drug Fund

RHB Regional Health Bureau

RHZE Rifampicin/Isoniazid/Pyrazinamide/Ethambutol

RRF Report and Requisition Form

SCMS Supply Chain Management Systems

SOP Standard Operating Procedures

TB Tuberculosis

USAID United States Agency for International Development

VRF Vaccine Request Form x

Foreword

Since its establishment in 2007, Ethiopian Pharmaceutical Supply Agency (EPSA) formerly known as Pharmaceutical Fund and Supply Agency (PFSA), the lead organization managing the health care supply chain of the country, has been working to ensure the availability, accessibility, and affordability of essential medications with appropriate quality, safety, and efficacy. To achieve these goals, EPSA— supported by its partners— has designed and implemented various innovative programs. The Integrated Pharmaceutical Logistics System (IPLS) is one of the major interventions undertaken to create a strong and unified healthcare supply chain, to connect all levels of the supply chain, and to provide accurate and timely data for decision making.

To initiate IPLS, a number of interventions were implemented, including (1) large-scale capacity building trainings for health facilities and higher levels, (2) a program of supportive supervision,

(3) physical improvements to warehouses and storerooms, and (4) implementing paper-based and automated logistic information management systems (LMIS). Currently, with this support, all public health hospitals and health centers are able to implement IPLS.

The survey results show the IPLS has already brought significant improvements to the supply chain in Ethiopia although much more remains to be done. The survey findings and recommendations provide valuable insight into the status of IPLS, including access to essential medicines, and the use of the LMIS formats and storage conditions. The information is expected to facilitate evidence-based planning thus contributing to a stronger and more efficient supply chain, increased medicine availability, and, ultimately, improved healthcare outcomes.

We strongly encourage all stakeholders involved in the healthcare supply chain to make the best use of this report in their planning and monitoring activities. The information will be particularly useful to government institutions and departments, health development and implementing partners, training and research institutions, as well as other national and international stakeholders.

Loko Abraham Marasi Mwencha

Director General Country Director

Pharmaceutical Supply Agency (EPSA) JSI Center for Health Logistics, AIDSFree xi

Acknowledgement

EPSA acknowledges with gratitude the financial and technical support from the JSI, AIDSFree project that made this work a reality.

Thanks also go to SART Consult, who recruited and trained the data collectors and implemented the survey.

We thank everybody involved in the design and implementation of this study; in particular, the data collectors and the informants who gave their time.

Finally, we thank the EPSA staffs, PSM-Chemonics Staffs, Regional Health Bureaus, Zonal Health

Departments, and health facilities for facilitating the overall data collection processes.

Loko Abreham

Director General

Ethiopian Pharmaceutical Supply Agency (EPSA) xii

EXECUTIVE SUMMARY

The Federal Ministry of Health (FMoH) has been working to ensure an efficient and high-performing healthcare supply chain that ensures equitable access to affordable medicines and related supplies for all Ethiopians. In recent years, significant progress has been made; however, various challenges remain including an inadequate supply of quality and affordable essential pharmaceuticals, poor storage conditions, and weak stock management which have resulted in high levels of waste and stock-outs.

In order to alleviate the challenges indicated above, PFSA (recently renamed as EPSA), in partnership with its partners developed and began implementing the IPLS in 2009. With the introduction of IPLS, EPSA established an integrated health commodity supply chain which accommodates all health program commodities; the system also tries to ensure that all levels – from health facilities to EPSA’s central stores – are connected with accurate and timely data for decision making.

Routine monitoring reports and a country-wide survey conducted in 2014 show IPLS is improving information recording and reporting, storage and distribution systems, as well as the availability of essential commodities at health facilities while also observing identified areas that need to be improved. One of the strong recommendations of the survey was the need for more monitoring and evaluation of IPLS including comprehensive quantitative surveys.

This survey was conducted from April to June 2018 to specifically assess pharmaceutical management practices and performance in IPLS at paper-based and electronic sites. The report is organized in four sections: section I presents introduction and objective, section II deals with survey methodology, section III deals with findings and discussion, and the last section (section

IV) presents conclusion and recommendations. The survey tool was adapted from the 2014 IPLS survey which was adopted from the USAID | DELIVER PROJECT’s Logistics Indicator Assessment

Tool (LIAT). Data collection took place from April to June 2018 from 456 randomly selected public health facilities. The data collection was conducted using smartphones after scripting the survey tool on SurveyToGo platform. SPSS version 21 and Microsoft Excel were used to manage the data.

The analysis was based on descriptive statistics, frequencies, percentages, cross tabulation, and averages. The results were disaggregated by health facility type (hospitals, health centers, and health posts). Hospital results were further disaggregated by level (primary, general and tertiary).

The results for the EPSA priority hospitals were also presented. A summary of the key findings, conclusions, and selected recommendations are presented below.

xiii

Key Findings

Only 59% of health posts had blank bin cards (an increase from 40% in 2015). Use of bin cards was even lower – the average use of bin cards for tracer items was 90% at hospitals, 55% at health centres and only 27% at health posts.

For almost every IPLS indicator, unsurprisingly, performance at health post level is poorer than at higher levels. For example, the use of bin cards is only 27% at health posts versus 55% at health centres; between 68% and 81% of bin cards at health centres were up-to-date, while for health posts the range was 55% to 65%; while bin card accuracy was in the range 42%-79% for health centres but only 33-56% for health posts.

Availability and use of stock keeping tools for vaccines is, generally speaking, lower than for IPLS commodities: for example, only 69% of health centres and hospitals had a VR, of those only 58% were using it (defined as an entry over the past six months), and of those using it only 63% were up-to-date and only 14% of VLs were accurate (41% had accurate or near accurate). VRFs for reporting and ordering vaccines were only available in 39% of health centres and of those only

34% were actually using them; the corresponding figures for the RRF were 85% and 83% respectively.

There are still shortages of pharmacy professionals working in the health sector. Despite major efforts to train more pharmacy professionals (both pharmacists and pharmacy technicians), 15% of health facilities (all of the health centres) had no pharmacy professionals on their staff. While significant investments have been made in training on IPLS and commodity management (in service, pre-service and recently online training), for a variety of reasons (service expansion, staff turnover) gaps remain. Nearly 24% of hospitals and health centres had no IPLS trained staff on their pharmacy units with in-service training remaining the most common training modality

(96%). At health posts, less than 10% of HEWs reported receiving informal OJT IPLS training, while 1% reported receiving pre-service training.

While sampling and methodological changes make it difficult for a valid comparison between the 2015 and 2018 surveys, the trends are concerning: while the percent of hospitals with acceptable storage conditions (meet 80% of criteria) increased from 43% to 71%, the percent of health centers declined from 63% to 45%; while the percent of health posts remained essentially unchanged from 29% to 27%.

xiv

In relation to Cold chain, only 65% of health centres had a functional refrigerator (97% had a refrigerator so nearly 1/3 health centres had a non-functioning refrigerator). Given that all health centres dispense vaccines, and many health posts rely on the cold chain at their associated health centre (only 18% of health posts had a functional refrigerator. The rest pick up and return vaccines to their health centre.

The health centres and hospitals surveyed showed just under half received direct delivery of program items (the exception is ART commodities since ART is typically offered in larger facilities, 96% of facilities offering ART have these delivered). For all other programs, just over half must go to their Woredas to collect medication.

More than half (54%) of health posts reported “on foot” as their main source of transport for health commodities. Another 16% cited public transport, and 4% animals, but less than 1% used bicycles.

Overall availability of most tracer items declined from the last survey (16 of 19 items). The decline was driven by a drop in availability at health centres – at hospitals, availability of most items actually improved. The availability at health facility level was about 79%. Availability of tracer items was, in almost every case, lower at health posts than at health centres.

The majority of health centres were overstocked for 10 of 13 items and hospitals for 9 of 12.

Automation seemed to have some impact here with 16% of items at automated sites stocked optimally versus only 9% of items at site managed using paper systems.

While inventory turnover seemed to improve over the three years of data examined (from 1 to

1.6 for RDF items, and 1.5 to 2.3 for program items), in all cases it remains less than optimal.

Wastage rate of Revolving Drug Fund (RDF) products was lower than program products for facilities. Generally, the wastage rate for both program and RDF pharmaceuticals was higher than 2% over the three years period.

Conclusions and Recommendations

While the availability of IPLS forms for all levels has improved since 2015, challenges remain particularly at health post level. If bin cards— the fundamental record keeping tool under IPLS— are not being used, then IPLS has a long way to go to become a national system.

Particular focus is needed for stock management at health posts. For almost every IPLS indicator, unsurprisingly, performance at health post level is poorer than at higher levels. Availability of tracer items was, in almost every case, lower at health posts than at health centres. Most of the xv attention on strengthening the supply chain continues to focus on higher levels and there is little investment in the last mile supply chain.

At health facilities, vaccines are managed separately from other items and, in general, inventory management is much weaker for vaccines than for IPLS items. Availability and use of stock keeping tools for vaccines is, generally speaking, lower than for IPLS commodities. A factor here may be that while IPLS formats are regularly printed centrally and distributed by EPSA, this normally does not happen for vaccine forms (VRFs were printed centrally and distributed in 2018 for the very first time). Facilities are expected to print forms when they need them resulting in a lack of standardization as the forms become modified or are simply not used. Another factor that should be looked at is form design. The VRF is complex, with many fields, and much harder to complete than the RRF. An interesting question would be if the amount of data required in the VRF actually means facilities are less likely to submit it?

There are still shortages of pharmacy professionals working in the health sector. Despite major efforts to train more pharmacy professionals (both pharmacists and pharmacy technicians), 15% of health facilities (all of the health centres) had no pharmacy professionals on their staff. This is concerning and deserves more research to identify root causes. Non pharmacy professionals do not receive any pre service training in commodity management, and are not a priority for IST.

Training gaps in supply chain management remain: While significant investments have been made in training on IPLS and commodity management (in service, pre-service and recently online training), for a variety of reasons (service expansion, staff turnover) gaps remain. The problem is even more pronounced at health posts. In-service training has been recently institutionalized for new HEW and the expectation is that this percentage will increase with time.

Storage conditions remain a major challenge for IPLS. While sampling and methodological changes make it difficult for a valid comparison between the 2015 and 2018 surveys, the trends are concerning: while the percent of hospitals with acceptable storage conditions (meet 80% of criteria) increased from 43% to 71%, the percent of health centers declined from 63% to 45%;

while the percent of health posts remained essentially unchanged from 29% to 27%. It does appear storage conditions at hospitals are receiving more attention than at health centers.

Cold chain conditions are a deep concern: significant number of health facilities assessed did not have functional refrigerators for vaccine storage at time of visit. Given that all health centers dispense vaccines, and many health posts rely on the cold chain at their associated health center

(only 18% of health posts had a functional refrigerator. The rest pick up and return vaccines to their health centre. This suggests the weak link in the cold chain is at the facility level and that maintenance and repairs needs strengthening.

xvi

Direct delivery has not progressed: Direct delivery of program commodities to facilities was one of the successes of IPLS. Currently, EPSA delivers to “accessible health facilities (those on “major routes” and, for the remainder, to their woreda. The representative sample of health centers and hospitals surveyed showed just under half received direct delivery of program items (the exception is ART commodities since ART is typically offered in larger facilities, 96% of facilities offering ART have these delivered). For all other programs, just over half must go to their

Woredas to collect medication. This is inefficient and pulls health staff away from their main role of providing healthcare. EPSA is currently considering increasing direct delivery from bimonthly to monthly delivery – while this will increase efficiency, consideration must be made for expanding direct delivery to more health centers and the resources this will require.

Health Extension Workers are still walking: while HEWS do collect supplies during routine visits to health centers, there is a need for more analysis to assess the burden on HEWs having to collect supplies and if there are steps that can be taken to support them.

Medicine availability is the ultimate indicator of supply chain performance: availability of 79% for a basket of tracer items is concerning and shows more work is needed to strengthen the supply chain. A well-functioning LMIS capable of providing either live or periodic inventory data would allow for a more reliable estimate of average availability and system performance. Of course, visibility of availability at facilities helps to not just measure performance but also helps directly improve performance. EPSA is working to improve data visibility from health facility level

Medicine availability is not the only problem: overstocking is also a problem. This may be partly due to concerns about stock outs leading facilities to order more than they need. Overall this contributes to an inefficient supply chain with resources tied up in slow-moving inventory, risk of expiry, and large storage areas needed to store excessive amounts of product. Automation seemed to have some impact here with 16% of items at automated sites stocked optimally versus only 9% of items at site managed using paper systems. Low inventory turns also point to an inefficient system.

Inventory turnover seemed to improve over the three years of data examined- in all cases it remains less than optimal : Such low inventory turnover (an inventory turn of 2 means the location has six months of stock at any moment) point to large dollar amounts tied up in inventory and an inefficient system, likely exacerbated by fear of stock outs in a vicious cycle:

levels over-order items because of fears of stock outs resulting in less funding for other items impacting availability and leading to more over-ordering.

Low order fill rates (less than 50% or most items): this is another symptom of an inefficient system that further perpetuates the problem. Facilities over order, don’t get what they order, and so they keep over-ordering, leading to both stock outs and overstocks.

xvii

To fix this problem will require a holistic effort that incorporates increased data visibility, systematic performance monitoring that includes KPIs that go beyond availability to include efficiency related KPIs like inventory turns, and order fill rate. Increased supply chain velocity— the time it takes for products to move through the supply chain from purchase to customers’ hands— will lead to a more responsive and efficient system, with fewer resources tied up in inventory. Other strategies EPSA should assess include ways to shorten procurement lead times, shorter distribution cycles (for example moving from the current bimonthly hub to facility distribution to monthly)

INTRODUCTION

The FMoH has been working to ensure an efficient and high-performing healthcare supply chain that ensures equitable access to affordable medication and related supplies for all Ethiopians.

While significant progress has been made, various challenges remain, including an inadequate supply of quality and affordable essential pharmaceuticals, poor storage conditions, and weak stock management, which have resulted in higher than desired levels of waste and stock-outs.

To strengthen the public sector healthcare supply chain, the FMoH initiated a comprehensive supply chain strategic planning process, emphasizing the integration of all products into one supply chain. In late 2006, the Ministry approved the Pharmaceutical Logistics Master Plan (PLMP), and in 2007, EPSA was established by Proclamation No. 553/2007, based on the PLMP. The agency’s mandate is to “avail affordable and quality pharmaceuticals sustainably to all public health facilities” (PSTP, 2015).

In order to effectively and efficiently execute this mandate and alleviate the challenges indicated above, EPSA, in collaboration with its partners, the USAID | DELIVER PROJECT, Supply Chain

Management Systems (SCMS), and others, developed and began implementing the IPLS in 2009.

With the introduction of IPLS, EPSA established an integrated health commodity supply chain that accommodates all health program commodities; the system also tries to ensure all levels— from health facilities to EPSA’s central stores— are connected with accurate and timely data for decision making. IPLS aims to streamline pharmaceuticals reporting and distribution by integrating the supply chain management of all types of pharmaceuticals (medication, medical supplies and equipment, and laboratory chemicals and reagents) for all programs. With three main components, policies and guidelines for LMIS, direct distribution to facilities, and inventory management of pharmaceuticals, IPLS measures progress and performance through a set of indicators for each component. These indicators assist EPSA in tracking supplies at all levels, and adherence on data use and reporting.

Routine monitoring reports and a country-wide survey conducted in 2014 show that IPLS was improving information recording and reporting, storage and distribution systems, as well as the availability of essential commodities at health facilities while also observing identified areas that need to be improved. One of the strong recommendations of the survey was the need for more monitoring and evaluation of IPLS including comprehensive and quantitative surveys in the future.

The latest survey was conducted from April to May 2018. The report is organized in four sections:

section I presents introduction and objective, section II deals with survey methodology, section III deals with findings and discussion, and the last section (section IV) presents a conclusion and recommendations.

OBJECTIVES

The general objective of the survey was to provide information on the status of IPLS implementation and measure its performance at public sector health facilities.

The specific objectives of the survey include:

To assess the availability of infrastructure, human resource, formats, tools and other enabling factors that are necessary for proper implementation of IPLS;

To assess select inventory and logistics system management practices, including the use of recording and reporting formats, inventory management, distribution and supervision;

To collect stock status information, including stock availability, stock-out duration, stock on hand, product expiration, and storage conditions;

To assess logistics system performance, through indicators like order fill rate and wastage rate;

To identify key issues and challenges in IPLS implementation to help determine the next steps needed for logistics system improvements; and

To provide data for comparative analysis of select key indicators since the 2014 survey.

METHODOLOGY

Study Population

The population of interest for this study was public sector service delivery points which included tertiary hospitals, general hospitals, primary hospitals, health centers, and health posts. Of the total hospitals in the sample, 22 were EPSA priority hospitals1, these are hospitals that receive special attention from EPSA due to their location, size, and importance.

Hospitals and health centers were considered a single population during sample calculation while health posts were treated as a separate population. As can be seen in Figure 1, health centers were under-represented in the final sample making up 81.6%, while it should have had a proportion of 93.0%. On the other hand, hospitals were over-represented, 7.2% and 11.1% for

EPSA priority hospitals and the other hospitals respectively, against 0.6% and 6.5% actual distribution. Due to such disproportionate representation and to reduce bias, data weights were applied during analysis. The national level estimations refer to hospitals and health centers unless and otherwise stated.

1 Adama Tertiary Hospital, ALERT Center, Amanuel Mental Specialized Hospital, Assela Teaching and Referral Hospital, Ayder Comprehensive

Specialized Hospital, Dagmawi Menelik Referral Hospital, Dilla University Referral Hospital, Felege Hiwot Comprehensive Specialized Hospital, Gandhi Memorial Hospital, Gondar University Specialized Hospital, Hawassa University Comprehensive Specialized Hospital, Hiwot Fana Specialized University Hospital, Jimma University Specialized Hospital, Rasdesta Damtew Memorial Hospital, St. Paul’s Hospital Millennium Medical College, St. Peter Specialized Hospital, Tikur Anbessa Specialized Hospital, Tirunesh Beijing General Hospital, Wolayta Sodo University Referral

Hospital, Yekatit 12 Hospital Medical College, Yirgalem General Hospital, and Zewditu Memorial Hospital.

Figure 1: Sample distribution from a commodity management perspective

Sample size determination

The sample calculation was based on the following formula2 which considered a single population for health facilities (hospitals and health centers). The main sample size was determined using expected proportions of study variables, margin of error, confidence level, and response rate.

n= 2

2 )1( d ppZ

Where:

n= sample size p= population proportion (e.g. availability of essential medicines3)

Z= associated Z value with the confidence level (90% confidence4 =1.645) d= margin of error5 (allowable error in the estimate or desired precision, e.g. ±0.05 that is

±5%)

Response rate (RR) = 95%

Since the total number of the health facilities was known, applying Finite Population Correction

(FPC), and considering a 95% response rate, the sample size became 266. The 266 samples were further stratified by health facility type and level per the natural proportion (see column four in

Table 1). However, the sample for the EPSA priority hospitals (1.5) and the other hospitals (17.3) were too small to generate indicative results by facility type or level; hence, these were boosted to 22 and 34 respectively.

2 Bhandarkar, P. L. & Wilkinson, T.S (1999). Methodology and Techniques of Social Research. Delhi: Himalaya Publishing House.

3 Mostly, a conservative estimate of 50 percent (0.50) is considered as it warrants the largest sample size.

4 Also called confidence coefficient, confidence level represents the likelihood that the confidence interval is to contain the true value.

5 Also referred as confidence interval, in statistics, a confidence interval is a particular kind of interval estimate of the true value of a population. Instead of estimating the true value by a single value, an interval likely to include the true value is given, e.g.

40±5%.

7.2 11.1

81.6

100.0

.6 6.5

93.0 100.0

PFSA priority hospitals (n=22)Other hospitals (n=34) Health centers (n=249) Total (n=305)

Achieved sample proportion (hospitals & health centers)

Table 1: Initial universe and original sample distribution by facility type

Facility types Universe Proportion (%) Original sample by proportion

EPSA priority hospitals 22 0.57% 1.5

Other hospitals 252 6.50% 17.3

Health centers 3616 92.93% 247.3

Total 3890 100.00% 266.0

Table 2 below, when it came to actual sample implementation, the total sample size became 305

(i.e., 22 EPSA priority hospitals, 34 hospitals, and 249 health centers). In addition, a total of 151 health posts (around 60.6% of the already sampled health centers) were selected as the total number of facilities planned in the scope were 456. The sample was further broken down proportionally by region. The health posts were treated separately as different sampling units during selection and analysis.

Table 2: Actual sample distribution by facility type and by region

Regions EPSA priority hospitals

Other hospitals

Health centers Total

Health posts

Total

(including health posts)

Oromia 3 10 96 109 57 166

SNNPR 4 6 47 57 29 86

Amhara 2 7 58 67 35 102

Somali - 2 12 14 7 21

Tigray 1 4 15 20 9 29

Afar - 1 6 7 4 11

Ben-Gum - 1 3 4 2 6

Gambella - 1 2 3 2 5

Addis Ababa 11 - 8 19 4 23

Harari 1 1 1 3 1 4

Dire Dawa - 1 1 2 1 3

Total 22 34 249 305 151 456

Sample Selection Procedures

EPSA priority hospitals, other hospitals, and health centers: All the available 22 EPSA priority hospitals were selected (i.e. census). The other hospitals and health centers were selected applying systematic random sampling technique which is a probability sampling approach that better ensures geographic representation. It was done for each region by calculating the sampling interval using the list of all available health facilities (i.e. sample frame) and the designated sample size for that particular region. Before applying the selection, the sample frame was sorted by zones followed by woreda.

Health posts: The sample for health posts accounted for 60.6% of the health centers sampled (i.e.

151 of 249). Since health posts are linked to health centers, the first step in selecting health posts required to randomly select the health centers using systematic sampling from the already selected 249 health centers. As a result, 151 health centers were selected from which health posts were selected. In each of the 151 selected health centers, all health posts served were listed and one was selected randomly using a random numbers table.

Survey tool

The survey tool was adapted from the 2014 IPLS survey which was adopted from the USAID |

DELIVER PROJECT’s LIAT tool. The 2014 tool was further refined and improved including adding questions to cover additional objectives. Feedback from data collectors training and pre-test were also utilized to further refine and improve the tool. The ordering of sections and questions was also revised to ensure better administration of the survey.

Data collection

Ten teams were set up to undertake the data collection. Each team comprised of a supervisor and two interviewers. All interviewers were pharmacy professionals as the research requires an in-depth understanding of the pharmaceutical logistics system. Prior experience in data collection was also considered in recruiting the interviewers.

All supervisors and interviewers were trained in Addis Ababa from March 28 to April 1, 2018. The major topics covered during the three-day training included: IPLS, objectives of the research, methodology (population, sample, and location), survey tool and administration procedure, ethical standards and quality assurance procedures. The session on the survey tool included dummy interviews using a scripted version of the survey tool on smartphones. The training was followed by a one-day pre-test. The pre-test was carried out in health facilities that were not part of the actual sample. The pre-test facilities were selected from Addis Ababa (Kirkos sub-city), Amhara (Chacha), and Oromia (Bishoftu, Fitche, Woliso, and Sebeta). The pre-test was done in two general hospitals and four health centers. A debrief session was held following the pre-test. The session helped revise the survey tool and its script as well as set out practical survey administration procedures.

Data collection took place from April 19 to June 4, 2018. JSI and EPSA staff joined the field teams to monitor adherence to quality standards and extend assistance when challenges occurred.

Data management

The data collection was conducted using smartphones. The survey tool was scripted on the Survey

ToGo platform. The scripted survey tool was tested rigorously prior to the fieldwork. Whenever there was internet connectivity, completed surveys were uploaded to a server. SPSS version 21 and Microsoft Excel were used to manage the data. The data was checked for logical consistency and accuracy.

The analysis was based on descriptive statistics: frequencies, percentages, cross tabulation, and averages. The results were disaggregated by health facility type (hospitals, health centers, and health posts). Hospital results were further disaggregated by level (tertiary, general, and primary).

The results for the EPSA priority hospitals were also presented.

Quality assurance

In order to ensure high-quality output, different quality assurance measures were put in place including:

All interviewers were pharmacy professionals; they were familiar with the pharmaceutical sector and the IPLS. This had significance in ensuring the survey tool and its administrative procedures were well understood and implemented.

The survey tool utilized lessons learned from the 2014 IPLS survey to improve it. Training and pre-test feedback were also utilized to refine it further.

The field team (supervisors and interviewers) were trained on the research and its methodology. They did dummy as well as actual pre-test interviews before embarking on the fieldwork.

The data was collected using the SurveyToGo platform which entails automatic skip patterns and range checks.

The GPS coordinates of the surveyed health facilities were captured automatically during the survey. Moreover, pictures of health facilities, their pharmacy stores, and cold chains were collected.

While fieldwork was in progress, reports on key indicators were generated on a weekly basis. Insights from the progress report helped guide the data cleaning and validation as well as analysis considerations.

The final data was checked for logical consistency and accuracy. Preliminary results were communicated with stakeholder and their inputs were utilized in preparing the final results.

Ethical considerations

Prior to the commencement of the fieldwork, regional health bureaus and the management of sampled health facilities were informed about the research. Respondents were briefed on the purpose, scope, and outputs of the research. They were informed that personal identifiers would never be used in any form while reporting, presenting and communicating the findings of the research and that data for individual health facilities would not be reported. Informed consent was obtained from respondents before interviewing them.

Limitations

The health post sample size is small compared to the total health posts in the country and therefore the results for health posts are not statistically significant.

While the comparison with the 2014 survey was important, a number of indicator definitions changed to be more comprehensive.

Survey Findings and Discussion

Facility description

Health facilities by level

A total of 456 facilities representing all the regions and the two city administrations were surveyed.

The majority, 54.6%, were health centers. Health posts and hospitals (tertiary, general, and primary) accounted for 33.1% and 12.2% of the sample respectively. Figure 2 illustrates the distribution of surveyed health facilities by level.

Figure 2: Distribution of sampled health facilities by level (%)

HCMIS FE (Dagu) implementation

Dagu (formerly known as HCMIS FE) is an inventory management system designed to manage daily transactions at health facilities. This system facilitates the use of standard operating logistics procedures including issue and receipts, ‘first to expire, first out’, and batch/expiry tracking. It also helps store and generates timely stock reports for decision making.

Ethiopia has been scaling up the implementation of Dagu at hospitals and health centers. The implementation status of Dagu (HCMIS FE) was assessed at hospitals and health centers. About

22% of sampled facilities reported implementing the system. More than 93% of tertiary and general hospitals and close to two-thirds of primary hospitals (63.3%) were implementing Dagu

(HCMIS FE) compared to just 18.5% of health centers. Figure 3 details the comparison of Dagu

(HCMIS FE) and paper-based/manual health facilities by level.

3.9 3.5

4.8

54.6

33.1 Tertiary hospital (n=18)

General hospital (n=16)

Primary hospital (n=22)

Health center (n=249)

Health post (n=151)

Figure 3: Health facilities by electronic (Dagu) and paper-based/manual

EPSA delivery modality

The survey examined the modality of health products delivery from higher levels to lower levels.

EPSA typically delivers products directly to health facilities every two months, or for hard to reach facilities it delivers to the woreda (district) with facilities picking up or woreda then delivering.

One of the objectives and responsibilities of EPSA is to scale up the direct delivery of health products to all health facilities. All of the tertiary, general and EPSA priority hospitals and 95.5% of primary hospitals reported getting program products directly from EPSA and among health centers, direct EPSA delivery stood at 47.8%.

Figure 4: Health facilities with direct EPSA delivery modality

5.60%

6.20%

36.40%

81.50%

77.90%

4.80%

94.40%

93.80%

63.60%

18.50%

22.10%

95.20%

Tertiary hospital (n=18)

General hospital (n=16)

Primary hospital (n=22)

Health center (n=249)

National

PFSA priority hospitals (n=22)

HCMIS (Dagu) implementation

Paper based/manual HCMIS

100.0% 100.0% 95.5%

47.8% 51.0%

100.0%

Tertiary hospital

(n=18)

General hospital

(n=16)

Primary hospital

(n=22)

Health center

(n=249)

National PFSA priority hospitals (n=22)

Infrastructure availability

Availability of infrastructure, namely (1) paved road6, (2) electricity, (3) water supply, (4) mobile/landline phone, (5) computer, (6) internet, and (7) backup generator was assessed to determine adequacy of existing infrastructure for effective pharmaceutical management and implementation of IPLS. This assessment, except for the availability of paved roads, was done at two levels: health facility in general and at the medical store(s). Internet and backup generator were reported as the two least available resources at the facility level with only 35.5% and 35.1% of hospitals and health centers reporting access. At medical stores, computer availability and water supply were the least available infrastructure with only 25.2% and 22.7% of hospitals and health centers reporting access, respectively.

It is worth noting the large differences between the two levels, health facility and medical store, for access to computers, water supply and backup generator. In all these cases access by the medical store was much less than that of the health facility in general, i.e. even though the resources were available for the health facility in general, they were not available at the medical store. Figure 5 has the details on the availability of infrastructure at health facilities and medical stores.

Figure 5: National level infrastructure availability

Facilities without operational electricity, internet, and mobile/wireless/landline phone access

Electricity, internet and mobile/wireless/landline phone services could be critical determinants for implementation of IPLS. Only 4.9% of hospitals and health centers at the national level had no

6 Only asked at facility level (not asked for pharmacy store).

24.1%

33.4%

22.7%

25.2%

81.7%

81.5%

35.1%

35.5%

63.0%

71.1%

82.4%

89.9%

91.2%

Operational backup generator

Internet access

Operational water supply

Operational computer

Mobile/wireless/land line phone

Operational elecrticity (main grid, generator, wind, or solar)

Paved road

Facility Store access to none of the three infrastructures (electricity, internet, and phone services). Access of health posts to the same three infrastructures is relatively low, though, with 27.2% of them not having access. Figure 6 illustrates the proportions of health facilities without access to the three infrastructure by health facility level.

Figure 6: Facilities without access to electricity, internet, and mobile phone

Logistics System Management Practices

As part of IPLS LMIS, various standardized recording and reporting formats are used at different levels of the healthcare supply chain.