Attachment_M_Illustrative_Quality_Checklist_from_Zambia.pdf

Page | 1

Attachment M: Zambian Health Center Quality Supervisory Checklist

DISCLAIMER: This illustrative checklist is being provided to Offerors of the USAID Haiti Services de Santé de Qualité pour Haïti for information purposes only. This checklist will likely form the basis for the Haitian quality checklist but may also be substantially revised by the Haiti MSPP for the Haitian context.

Purpose of this checklist The purpose of this checklist is to obtain, once per quarter, a summary quality score for a Zambian Health Center. This quality score will be used in the performance framework for Results-Based Financing (RBF). The Fee-For-Service system (each defined service carries a certain bonus fee) is influenced by this quality score. If for instance the quality is 78%, then only 78% of the quarterly bonus fees will be paid to the health facility. This system is designed to not only stimulate the quantity of certain services provided, but to also ensure a high level of quality. Also, certain desirable elements of quality provision to the Zambian people cannot be measured through single indicators, but can only be captured by so-called ‘proxy indicators’: indicators that measure something seemingly different, as they attempt to measure the process of care. An example is the desirable fact that pregnant women should be provided Ferro sulfate to combat anemia. One cannot directly purchase this indicator, as the verification would be challenging. The moment one decides to purchase a certain service (and not other services), the reporting on the said service tends to become unreliable, and the techniques that we apply – community client surveys- are not able to distinguish between Ferro sulfate and any other tablet. However, we can measure (i) the availability of Ferro sulfate in the pharmacy; (ii) the stock keeping and record keeping; (iii) correct HMIS usage;

(iv) the availability of equipment and other supplies related to antenatal care services and (v) many other quality elements across other services that offer a balanced view of the general level of functioning (otherwise said: the ‘quality of services’).

Methodology for applying the Quality checklist Once per quarter, each health center is subject to a quality checklist by a health officer, mandated to do so. The mandate is provided by the District RBF Steering Committee. The travel request, travel arrangements (motorcycle; car and per diem) should be arranged through the usual channels. The quarterly quality checklist needs to be completed before …….. day following the third month. The checklists are filed in a separate folder in the DHO. Results are presented and discussed during the quarterly RBF Steering Committee meetings. This quality check should be separate from normal ‘routine’ supervisory visits which have a formative (training) character. It goes without saying that the controlling health officer should provide constructive feedback after the check. But the health officer may not advice the health center staff what to do and then score activities afterwards.

Page | 2

Methodology for the independent counter-verification of reported results (after payment for results) Routine counter-verification of reported results will be done through an independent and transparent mechanism. A protocol with a multi-stage random sampling method will be applied. A deviation between the reported findings and the findings from the counter-verification – closer defined in the protocol – will be followed by an in-depth investigation and sanctions for the involved staff, which can include, but are not limited to, immediate exclusion from the RBF incentive scheme.

Proposed Weighting for the Quality Score Card [Weighting is indicative and meant to elicit change. For changing: first look at the relative weights in the table below, and re-adjust according to the group of expert preference – Delphi-, then adapt the individual indicator weights for the various services in the Quality Check List to fit the summary weights]

No SERVICE WEIGHT

1 Curative Care 35

2 ANC 50

3 FP 57

4 EPI 28

5 Delivery Room 65

6 HIV 16

7 Supply Management 21

8 General Management 20

9 HMIS 18

10 Community Participation 9

Total 319

Page | 3

The Zambia Health Center Quality Checklist [This is a draft, and needs to be scrutinized and adapted to fit local realities. One needs to check (i) the actual substance of the indicators and composite criteria; (ii) check the weighting allocated to the indicators and composite criteria and (iii) agree on the form. The form can be improved by listing each composite criterion on its own line; however, this will take up more space. Experience with the approach below is that with field testing, and adequate training, it can be applied reliably by controllers. The checklist is sculpted on the assumption that it will be applied to a relatively small Zambian rural health center, with a typical staff of one Clinical Officer or Zambian Enrolled Nurse, one or two Classified Daily Employees, and one or several Community Health Workers attached to the health center.

This checklist is not supposed to be a final product: typically it is modified regularly, incorporating lessons learned from the end-users, incorporating changing protocols and norms and values. Moreover, it is changed to regularly ‘put the barrier higher’. After the majority of health centers reach towards 90% or over, one needs either to adapt the checklist or think about the reliability of the ME. Typically national norms and values are many. It is the purpose of this checklist to incorporate those that are deemed more important than others.

This is why the creation, and continuous change, of this checklist is more of an art than a science. Our Zambian colleagues are invited to take a critical look at it, and adapt it to their situation.]

Zambia Health Center Quality Checklist

1. Curative Care

No Checklist Item Criteria Points

1 Equipment available in the treatment room and in working condition 1) thermometer 2) blood pressure cuff 3) stethoscope 4) otoscope 5) unused non-sterile gloves 6) adult scale 7) tongue depressor 8) examination table 9) bowl for washing hands with soap, filled with water and a clean towel for drying hands

Treatment room equipped with 9 functional materials = 9

One material missing or non-functional = 5 More than one material missing or non-functional = 0

2 Privacy: Individual treatment room with curtains or painted windows, room divider (if shared room), doors that close

Assured privacy = 3

3 Documentation for consultation 5 documents present in

Page | 4 available to provider: 1) Nursing flowchart 2) Sexual Transmissible Diseases flowchart 3) National Malaria Program fever flowchart 4) National guidelines for management of malaria in Zambia 5) TB Guide consultation room = 3 1 document missing = 0

4 Treatment of 5 consecutive cases of children under age 5 according to flowchart or protocol

Each correct entry and exit points according to flowchart or protocol =4

Subtotal Curative Services (x/35) *100%

2. ANC

No Checklist Item Criteria Points

5 First ANC visit, first such client:

Questioning on (using appropriate language): 1) gynecological and obstetrical history (gravida/para/abortus) including tetanus vaccination status 2) convulsions 3) medical and surgical history: i) diabetes, ii) heart disease, iii) hypertension, iv) kidney disease, v)TB, and vi) asthma, 4) HIV testing (with guidance if appropriate)

First case fulfilling all criteria

= 5

First case with even one unmet criterion = 0. If subsequent cases, observed accidentally, do not fulfill all criteria = 0

6 First ANC visit, first such client, Physical examination 1) weight 2) height 3) blood pressure 4) breast examination and 5) check for edema

First case fulfilling all criteria

= 5

First case with even one unmet criterion = 0. If subsequent cases, observed accidentally, do not fulfill all criteria = 0

7 First ANC visit, first such client:

Obstetric examination: 1) height of uterus 2) presentation (from 36 weeks) 3) fetal heartbeat (from 20 weeks) 4)

First case fulfilling all criteria

= 5

First case with even one unmet

Page | 5 vaginal touch criterion = 0. If subsequent cases, observed accidentally, do not fulfill all criteria = 0

8 First ANC visit, first such client:

Administration of tetanus vaccine according to guidelines: 1) correct intervals 2) unexpired vaccine 3) storage at vaccination site in compliance with regulations 4) means of administering vaccine corresponds to instructions on vial and injection site is in compliance with regulations

First case fulfilling all criteria

= 5

First case with even one unmet criterion = 0. If subsequent cases, observed accidentally, do not fulfill all criteria = 0

9 First ANC visit, first such client:

Correct prescription of: 1) iron for women who are HIV-negative or whose HIV status is unknown 2) folic acid 3) Mebendazole (from second trimester) 4) Fansidar DOT (from 4th month) – 5) insecticide-treated mosquito nets

First case fulfilling all criteria

= 5

First case with even one unmet criterion = 0. If subsequent cases, observed accidentally, do not fulfill all criteria = 0

10 First ANC visit, first such client:

Risk-based management: 1) risk factors identified 2) appropriate decision taken according to ANC chart 3) information communicated to patient

First case fulfilling all criteria

= 5

First case with even one unmet criterion = 0. If subsequent cases, observed accidentally, do not fulfill all criteria = 0

11 First Any Subsequent ANC visit:

Questioning: Complaints about current pregnancy

Questioning done = 2.5

If not done = 0, If subsequent cases, observed accidentally, do not do this = 0

2.5

12 First Any Subsequent ANC visit:

Physical examination: 1) weight 2) blood pressure 3) breast examination and

4) check for edema

One examination of a case with all relevant elements = 2.5 One examination of a case with even one relevant element missing = 0, If subsequent cases, observed accidentally, do

2.5

Page | 6 not do this = 0

13 First Any Subsequent ANC visit:

Obstetric examination: 1) height of uterus 2) presentation (from 36 weeks) 3) fetal heartbeat (from 20 weeks)

One examination of a case with all relevant elements = 2.5 One examination of a case with even one relevant element missing = 0, If subsequent cases, observed accidentally, do not do this = 0

2.5

14 First Any Subsequent ANC visit:

Administration of tetanus vaccine according to guidelines: 1) correct intervals 2) unexpired vaccine 3) conservation according to local rules 4) means of administering vaccine corresponds to instructions on vial and injection site is in compliance with regulations

One case of administering a vaccine fulfilling all criteria = 2.5

One case with one unmet criterion = 0, If subsequent cases, observed accidentally, do not do this = 0

2.5

15 First Any Subsequent ANC visit:

Correct prescription of: 1) iron for women who are HIV-negative or whose HIV status is unknown 2) folic acid 3) Mebendazole (from second trimester) 4) Fansidar DOT (from 4th month)

One case with prescriptions given correctly = 5

One case with even one incorrect prescription = 0, If subsequent cases, observed accidentally, do not do this = 0

16 First Any Subsequent ANC visit:

Management of cases with complications: 1) complications identified 2) appropriate decision taken according to flowchart 3) information communicated to patient

First case fulfilling all criteria

= 5

First case with even one unmet criterion = 0. If subsequent cases, observed accidentally, do not fulfill all criteria = 0

Subtotal Antenatal Consultation Services (x/50)* 100%

3. FP

No Checklist Item Criteria Points

Page | 7

17 Direct Observation of the Room and Supplies: Contraceptive methods:

1. Pills

2. Injectable contraceptives

3. Implant

4. IUD with uterine forceps and hysterometer

5. Condoms

6. Collar

A) Availability of contraceptive with theoretical stock corresponding to physical stock; and B) alert and security thresholds determined and respected.

One contraceptive method fulfilling both criteria A and B = 2

A contraceptive method with even one unmet criterion = 0

18 Analysis of ten randomly chosen FP clients from the past three months:

Justification of methods recommended, used, and prescribed compared to methods indicated on the basis of questioning, history, and physical examination

Correct justification = 3 Unjustified = 0

19 Verification and Appointment: Scheduled appointment of these 10 cases.

Yes = 1.5

No = 0

Subtotal Family Planning Services (x/57) * 100%

4. EPI

No Checklist Item Criteria Points

20 EPI: Available and functional equipment and supplies: 1) refrigerator

2) cold accumulators 3) insulated foam cooler 4) spare parts (wicks and glass containers for petrol refrigerator) 5) petrol reserve (5 liters minimum) and/or generator 6) thermometer

All equipment and supplies available and functional = 2

Even one piece of equipment or supply missing or non-functional = 0

21 Availability of vaccines and diluents:

BCG

Polio

Availability of all antigens and diluents fulfilling both criteria

Page | 8

Pentavalent Tetanus Measles Diluting agent

1) Physical presence of unexpired, labeled antigens 2) No interruption of supply during past three months

= 2

Expiry or disruption of supply of one antigen or diluting agent = 0

22 Storage of vaccines: 1) Vaccines carefully arranged in refrigerator sections (frozen and non-frozen vaccines) 2) No products other than vaccines and accumulators stored in refrigerator

Storage fulfilling both criteria = 2.5

Storage with even one unmet criterion = 0

2.5

23 Available consumables and printed forms: 1) self-blocking syringes 2) dilution syringes (2 and 5 ml)

3) receptacles 4) absorbent cotton 5) vaccination charts 6) control sheets 7) record of vaccinations

Availability of all items= 2

Even one item missing = 0 2

24 Cold chain: 1) Max. and min.

temperatures of refrigerator between +2*C and +8*C, 2) Unbroken cold chain during previous three months.

Chain fulfilling both criteria = 3

Even one day of interruption= 0

25 Management of vaccine supply: 1) theoretical supply of vaccines corresponds to physical supply 2) alert and security thresholds determined and respected.

Management fulfilling both criteria = 3

Even one unmet criterion = 0

26 Direct observation of five consecutive children in the vaccination program:

Growth monitoring chart checked to determine whether vaccination is due

Checked for each case = 0.5 2.5

27 Systematic check for BCG scar Checked for each case = 0.5 2.5

28 Preparation of vaccines: 1) vaccine vial monitor (VVM) in good condition 2) correct dilution technique 3) use of self-blocking syringe 4) correct dosage

All five cases fulfilling all criteria = 2

One case with even one unmet criterion = 0

Page | 9

29 Injection: 1) Cleansing of injection site with cotton 2) Correct injection methods and sites

One case fulfilling both criteria = 0.5

One case with even one unmet criterion = 0

2.5

30 Side effects systematically noted Yes for one case = 0.5

No for one case = 0 2.5

31 Registration completed: 1) on individual chart 2) on growth chart 3) in register 4) on tracking form

One case with registration fulfilling all criteria = 0.5

One case with even one unmet criterion = 0

2.5

Subtotal EPI (x/28)*100%

5. Delivery Room

No Checklist Item Criteria Points

32 Privacy: Curtains or painted windows, room divider (if shared room), doors that close

Assured privacy = 7 7

33 Available and functional equipment and supplies: 1) adjustable, clean delivery table 2) at least 3 sterilized instrument boxes (with needle holder, two Kocher clamps, toothed forceps, two pairs of scissors) 3) neonatal suction devise 4) obstetrical stethoscope 5) suture thread, 6) light source 7) infant weighing scale 8) sterilizing drum 9) ophthalmic ointment 10) gauze drum 11) plastic apron 12) local anesthesia (at least 50ml in reserve) 13) AMBU bag for neonates 14) theater boots 15) mask 16) goggles 17) unused and non-torn surgical gloves 18) umbilical cord clamp

One material available and functional = 1

If even one material (1) to (5) is unavailable or non-functional =

34 Analysis of 10 randomly selected partograms: 1) Partogram filled out

One partogram fulfilling 3 criteria = 4

Page | 10 according to the rules 2) Decision made/documented if alert line is passed within one hour 3) Delivery by qualified staff (at least a nurse A2)

One partogram with even one unmet criterion = 0

Subtotal Delivery Room (x/65) * 100%

6. HIV

No Checklist Item Criteria Points

35 VCT/PMTCT: Equipment and supplies. 1) Equipment and supplies meet MOH standards

(i) one desk (ii) three chairs (iii) bookshelf (iv) single-use receptacle (v) materials for taking blood samples (tubes, needles, gloves, adaptor, tourniquet, tube holder, stopper) (vi) lab forms

2) IEC/BCC supplies available on the table

(i) Box of condoms (at least 10 condoms) (ii) model of a penis, (iii) HIV/AIDS booklets/leaflets pertaining to VCT and PMTCT [

Criteria 1) and 2) fulfilled = 16

One criterion unmet = 0

Subtotal HIV (x/16) * 100%

7. Supply Management

No Checklist Item Criteria Points

36 Pharmacy premises in accordance with regulations: 1) shelves 2) ventilated premises 3) protection against direct sunlight 4) protection against theft.

Premises fulfilling all criteria = Even one unmet criterion = 0

Cleanliness of pharmacy (no dust on shelves and products, no cobwebs)

Cleanliness assured = 2 2

37 Stocking in accordance with regulations: 1) products arranged on shelves, not on floor 2) logically arranged products (alphabetical order or by type of

Stocking fulfilling all criteria = 5 Even one unmet criterion = 0

Page | 11 therapy) 3) on basis of expiry date 4) with signs on shelves according to International Common Denomination (generic names) 5) agreement between theoretical and physical stock

38 Management of tools:

Inventory card Order forms

1) Presence of tool 2) Filled out according to regulations

One tool fulfilling both criteria = 0.5 One tool with even one unmet criterion = 0

39 Availability of tracer drugs and consumables: 1) all molecules and consumables available 2) No interruption of supply since last assessment.

All tracer drugs and consumables available = 6 Disruption of even one drug or consumable = 0

40 Correct procedure observed for destruction of outdated products : 1) inventory card for outdated products

2) Acknowledgment of outdated drugs and reagents received by the hospital.

Procedure fulfilling both criteria = 2 Even one unmet criterion = 0

Subtotal Supply Chain Management (x/21) * 100%

8. General Management

No Checklist Item Criteria Points

41 Available minutes of the three previous monthly meetings of the health center management committee. Minutes should contain: 1) date, time of opening and closing of meeting 2) agenda items 3) signed list of persons present 4) description of the meeting a) adoption of minutes of previous meeting; b) review of recommendations or actions of previous meeting; c) topics discussed, including the following: i) description of topic; ii) decisions or recommendations;

iii) execution timetable; iv) person responsible; v) observation 5) signatures

Minutes fulfilling all criteria =

Minutes with even one criterion unmet = 0

Page | 12 of chairman and secretary.

42 Inventory of equipment and supplies updated monthly

All inventories present and up-to-date = 1 Missing or obsolete inventory =

43 Services available with staff on duty 24/7, including holidays

Posted duty list and schedule =

44 Reception: 1) Covered waiting rooms complete with chairs or benches; 2) A system of waiting room numbers/tickets and guidance in place 3) Staff member conducting triage according to seriousness of problem and waiting room number/ticket

One element present = 1 (max.

3 elements)

45 Cleanliness of rooms, halls, and grounds: 1) presence of trash receptacles (in waiting room and corridor) 2) no loose trash;3) receptacles for syringes present in treatment rooms 4) grounds surrounding health center entirely cleared of weeds and stagnant water drained

Each cleanliness criterion fulfilled = 0.5

46 No organic waste, syringes, or dangerous products in halls, rooms, or any other location on the grounds of the health center that are easily accessible to the public

Yes = 1 Presence of organic waste or used syringes, needles or used bandages = 5-point penalty

47 Functioning incinerator or disposal pit:

operated and used according to the rules (triage and destruction, etc.)

Functioning incinerator or disposal pit used in accordance with regulations = 1

48 Placenta pit with slab and cover Yes = 1 1

49 Availability of water source (running water or well, pump, or water tower/tank

Available water source = 1 1

50 Water dispensers available in consultation rooms, in delivery room and near latrines

Available water dispensers = 1 1

51 Presence of latrines and showers 1) usable; 2) no organic matter within or outside; 3) door that closes from the

All latrines fulfill criteria = 1 All showers fulfill criteria = 1

Page | 13 inside; 4) covered pit (for latrines)

52 Available and functional sterilization materials: pressure sterilizer

Pressure sterilizer available and functional = 1

53 Clean, neat uniforms worn by all full-time or part-time staff. Staff includes CDE and CHWs employed by the HC.

Uniforms worn by all staff = 1 1

Subtotal General Management (x/20) * 100%

9. HMIS

No Checklist Item Criteria Points

45 Standard RBF Registers available and columns filled correctly, legibly and completely: 1) Curative Care; 2) FP;

3)ANC; 4) EPI; 5) VCT/PMTCT

Columns as in the RBF handbook, completely and legibly filled for each standard register. Each register well filled = 10 if even 1 RBF register not well-filled = 0 Definition not well filled: (i) columns not according to set format, (ii) missing information over the past period (glancing through) and (iii) systematic illegible information

46 HIS report transmitted during the period (by the 5th working day of following month)

Acknowledgment of receipt of a monthly HMIS report during the period = 1

47 Monthly analysis report of Health Information System (HIS) data on priority problems (Vaccination, FP, maternity, malaria, HIV and other priority diseases in the region) containing: 1) Graph or table of data 2) Comments on implementation of activities as compared to targets. This can be in a small, dedicated register book.

Each Monthly Analysis report of data on 6 priority problems containing all elements = 1

Report with even one element missing = 0

48 Data concordance between 3 randomly selected HIS indicators and activity

Concordance of all three indicators = 2

Page | 14 records Even one non-concordant indicator = 0

Subtotal HMIS (x/18) * 100%

10. Community Participation

No Checklist Item Criteria Points

49 Available minutes of past three monthly meetings on analysis of community data with community health workers. Minutes should contain: 1) date, time of opening and closing of meeting 2) agenda items

3) signed list of persons present 4) description of the meeting a) adoption of minutes of previous meeting; b) review of recommendations or actions of previous meeting; c) topics discussed, including the following: i) description of topic; ii) decisions or recommendations;

iii) execution timetable; iv) person responsible; v) observation 5) signatures of chairman and secretary.

Minutes fulfilling all criteria

= 3

Minutes with even one criterion unmet = 0

Subtotal Community Participation (x/9) * 100%

Total Quarterly Quality Score (number of points obtained/319) * 100%

Optional space for handwritten comments or observations. Need space for signature staff and controlling officer.