Lancet Global Health_Appendix1.pdf

Supplementary appendix 1 This appendix formed part of the original submission and has been peer reviewed.

We post it as supplied by the authors.

Supplement to: Kruk ME, Gage A, Arsenault C, et al. High-quality health systems in the Sustainable Development Goals era: time for a revolution. Lancet Glob Health 2018;

published online Sept 5. http://dx.doi.org/10.1016/S2214-109X(18)30386-3.

High-quality health systems in the Sustainable Development Goals era: time for a revolution

Appendix 1: Methods

Table of Contents

Section 2: What quality of care are people receiving in LMICs today?

1. Country income groups

2. Datasets analyzed

3. HQSS Commission-led Survey of User Experience in 12 LMICs

4. Definitions of quality measures included in section 2

5. Panel 2: Beyond the numbers – experiences in the health system: review

6. Adjusting for quality in the Lives Saved Tool

7. Why quality of maternal mental health care may suffer for vulnerable groups: An illustration with perinatal depression care in primary care setting in Nigeria (Panel 4)

Section 3: The ethical basis of high-quality health systems

1. Social accountability for health quality review

2. Actions to support legal and social accountability, review (Panel 7)

Section 4: Measuring health system quality

1. Quality subdomains in global, cross-national, and national measurement sets, indicator review (Figure 12)

2. Innovation in patient experience and outcome measure: mobile data collection of patient reported outcome measures among women in Nairobi, Kenya (Panel 11)

Section 5: Improving health systems at scale

1. Intervention typology analysis

2. Types of interventions and levels targeted to improve quality of primary health care: a scoping review (Figure 15)

3. Quality governance in a pluralistic health system: Mexican experience and challenges .. 30

4. Scoping review of reviews on accreditation

5. Governing for quality: a stakeholder consultation in five LMICs

6. Improving delivery care quality at scale: Modelling service redesign in six countries

7. Quality of care among recent clinical graduates (first 3 years of practice) in nine LMICs 34

8. Update of a systematic review of the effectiveness of strategies to improve health care provider performance in low- and middle-income countries (Table 3)

9. Systematic review of quality improvement collaboratives in LMICs

10. Improving health care provider performance in LMICs: how strategy effectiveness changes over time

Additional methods and notes

1. Reference list for figures, panels and tables

2. Additional works consulted

4. HQSS National Commissions

5. Advisory groups

6. Additional contributors

Section 2: What quality of care are people receiving in LMICs today?

1. Country income groups

The Commission’s scope is all low- and middle-income countries as classified by the World

Bank, listed below. Data presented in Section 2 used the country’s income group specific to the survey year based on World Bank classification, which may differ from the current classification

(as of January 2018) below.

Country Income Group

Afghanistan Low income

Albania Upper middle income

Algeria Upper middle income

American Samoa Upper middle income

Angola Lower middle income

Argentina Upper middle income

Armenia Lower middle income

Azerbaijan Upper middle income

Bangladesh Lower middle income

Belarus Upper middle income

Belize Upper middle income

Benin Low income

Bhutan Lower middle income

Bolivia Lower middle income

Bosnia and Herzegovina Upper middle income

Botswana Upper middle income

Brazil Upper middle income

Bulgaria Upper middle income

Burkina Faso Low income

Burundi Low income

Cabo Verde Lower middle income

Cambodia Lower middle income

Cameroon Lower middle income

Central African Republic Low income

Chad Low income

China Upper middle income

Colombia Upper middle income

Comoros Low income

Congo Lower middle income

Costa Rica Upper middle income

Côte d'Ivoire Lower middle income

Croatia Upper middle income

Cuba Upper middle income

Dem. Rep. Congo Low income

Djibouti Lower middle income

Dominica Upper middle income

Dominican Republic Upper middle income

Ecuador Upper middle income

Egypt Lower middle income

El Salvador Lower middle income

Equatorial Guinea Upper middle income

Eritrea Low income

Ethiopia Low income

Fiji Upper middle income

Gabon Upper middle income

The Gambia Low income

Georgia Lower middle income

Ghana Lower middle income

Grenada Upper middle income

Guatemala Lower middle income

Guinea Low income

Guinea-Bissau Low income

Guyana Upper middle income

Haiti Low income

Honduras Lower middle income

India Lower middle income

Indonesia Lower middle income

Iran Upper middle income

Iraq Upper middle income

Jamaica Upper middle income

Jordan Lower middle income

Kazakhstan Upper middle income

Kenya Lower middle income

Kiribati Lower middle income

Dem. People's Rep. Korea Low income

Kosovo Lower middle income

Kyrgyzstan Lower middle income

Laos Lower middle income

Lebanon Upper middle income

Lesotho Lower middle income

Liberia Low income

Libya Upper middle income

Macedonia Upper middle income

Madagascar Low income

Malawi Low income

Malaysia Upper middle income

Maldives Upper middle income

Mali Low income

Marshall Islands Upper middle income

Mauritania Lower middle income

Mauritius Upper middle income

Mexico Upper middle income

Micronesia Lower middle income

Moldova Lower middle income

Mongolia Lower middle income

Montenegro Upper middle income

Morocco Lower middle income

Mozambique Low income

Myanmar Lower middle income

Namibia Upper middle income

Nauru Upper middle income

Nepal Low income

Nicaragua Lower middle income

Niger Low income

Nigeria Lower middle income

Pakistan Lower middle income

Palestine Lower middle income

Panama Upper middle income

Papua New Guinea Lower middle income

Paraguay Upper middle income

Peru Upper middle income

Philippines Lower middle income

Romania Upper middle income

Russia Upper middle income

Rwanda Low income

Samoa Upper middle income

São Tomé and Principe Lower middle income

Senegal Low income

Serbia Upper middle income

Sierra Leone Low income

Solomon Islands Lower middle income

Somalia Low income

South Africa Upper middle income

South Sudan Low income

Sri Lanka Lower middle income

St. Lucia Upper middle income

St. Vincent and the Grenadines Upper middle income

Sudan Lower middle income

Suriname Upper middle income

Swaziland Lower middle income

Syria Lower middle income

Tajikistan Lower middle income

Tanzania Low income

Thailand Upper middle income

Timor-Leste Lower middle income

Togo Low income

Tonga Upper middle income

Tunisia Lower middle income

Turkey Upper middle income

Turkmenistan Upper middle income

Tuvalu Upper middle income

Uganda Low income

Ukraine Lower middle income

Uzbekistan Lower middle income

Vanuatu Lower middle income

Venezuela Upper middle income

Vietnam Lower middle income

Yemen Lower middle income

Zambia Lower middle income

Zimbabwe Low income

2. Datasets analyzed

Quality estimates in Section 2 draw on cross-national data from health facility, household, telephone and internet surveys collected between 2007 and 2017. This table presents all surveys for which microdata was analyzed in this Commission. Unless indicated below, we used survey sampling weights. Aggregation across countries are unweighted means, so that each country weighs equally.

Dataset Available from Countries and years Type of data used Sampling and notes

Service Provision

Assessment

Survey (SPA) https://dhsprogram.com/

Ethiopia 2014, Haiti 2013, Kenya 2010, Malawi

2013-14, Namibia 2009, Nepal 2015, Rwanda 2007, Senegal 2015-16, Tanzania 2014-15, Uganda 2007

Direct observations and exit interviews

Nationally representative samples of health facilities, censuses or near-censuses.

Results Based

Financing impact evaluations baseline health facility surveys

(RBF)

http://microdata.worldbank

.org/

Burkina Faso 2013, Central African Republic 2012, Cameroon 2011, Republic of the Congo 2014, Democratic Republic of the Congo 2015, Kyrgyzstan 2012-13, Nigeria 2013, Tajikistan

2014-15

Direct observations

RBF health facility assessment sampling varied by country but generally included all hospitals and a random sample of primary health care facilities in the districts selected for the RBF evaluation. Estimates from the RBF datasets are unweighted.

World Bank

Service Delivery

Indicators (SDI) https://data.worldbank.org/ data-catalog/service-delivery-indicators

Kenya 2012, Nigeria 2013, Tanzania 2013-14, Togo

2013, Uganda 2013

Clinical vignettes

Demographic and

Health Surveys

(DHS)

https://dhsprogram.com/

Afghanistan 2015, Albania 2008-2009, Armenia

2010, Bangladesh 2014, Benin 2011-2012, Bolivia

2008, Burkina Faso 2010, Burundi 2010, Cambodia

2014, Cameroon 2011, Chad 2014-2015, Colombia

2015, Comoros 2012, Republic of Congo 2011-

2012, Cote D'Ivoire 2012, Dominican Republic

2013, Democratic Republic of the Congo 2014, Egypt 2014, Ethiopia 2016, Gabon 2012, Ghana

2014, Guatemala 2014-2015, Guinea 2012, Guyana

2009, Haiti 2012, Honduras 2011-2012, Indonesia

2012, Jordan 2012, Kenya 2014, Kyrgyzstan 2012, Lesotho 2014, Liberia 2013, Madagascar 2009, Malawi 2015-2016, Maldives 2009, Mali 2013, Mozambique 2011, Namibia 2013, Nepal 2011, Niger 2012, Nigeria 2013, Pakistan 2012-2013, Population survey

We used the most recent DHS or

MICS survey available if a country had both within the last ten years.

Peru 2009-2012, Philippines 2013, Rwanda 2014-

2015, Senegal 2013-2016, Sierra Leone 2013, Swaziland 2006-2007, Tajikistan 2012, Tanzania

2015-2016, Timor Leste 2010, Togo 2013-2014, Uganda 2011, Ukraine 2007, Yemen 2013, Zambia

2013-2014, Zimbabwe 2015

UNICEF Multiple

Indicator Cluster

Surveys (MICS) http://mics.unicef.org/

Algeria 2012-13, Argentina 2012-13, Belarus 2012, Belize 2011, Bhutan 2010, Bosnia and Herzegovina

2011-12, Central African Republic 2010, Costa

Rica 2011, Cuba 2014, El Salvador 2014, Guinea

Bissau 2014, Iraq 2011, Jamaica 2011, Kazakhstan

2015, Kosovo 2013-14, Laos 2011-12, Macedonia

2011, Mauritania 2011, Mexico 2015, Moldova

2012, Mongolia 2013-14, Montenegro 2013, Palestine 2014, Panama 2013, Saint Lucia 2012, Sao Tome and Principe 2014, Serbia 2014, South

Sudan 2010, Sudan 2014, Suriname 2010, Thailand

2012-13, Tunisia 2011-12, Turkmenistan 2015-16, Vietnam 2013-14

Population survey

We used the most recent DHS or

MICS survey available if a country had both within the last ten years.

WHO Study on global AGEing and adult health

(SAGE)

http://www.who.int/health info/sage/en/

China 2007-2010, Ghana 2007-2008, India 2007-

08, Mexico 2009-10, Russia 2007-2010, South

Africa 2007-08

Population survey

Commonwealth

Fund International

Health Policy

Survey (CWF) http://www.commonwealth fund.org/interactives-and-data

Australia, Canada, New Zealand, United Kingdom, United States, Germany, Netherlands, France, Norway, Sweden, Switzerland

2013 for all countries survey

Used as high-income comparison only.

Inter-American

Development

Bank primary care survey (IDB)

N/A

Brazil, Colombia, El Salvador, Jamaica, Mexico, Panama

2013 for all countries

HQSS

Commission-led internet survey

(HQSS)

N/A

Argentina, China, Ghana, India, Indonesia, Kenya, Lebanon, Mexico, Morocco, Nigeria, Senegal, South Africa

2017 for all countries survey

Internet survey of user experience in

12 LMICs conducted for the

Commission by Riwi Corporation.

See below for further detail.

Mexican Institute of Social Security N/A Mexico 2016

Electronic health records

The Mexican Institute of Social

Security (IMSS) is the largest Health

(IMSS) system in Mexico; it provides healthcare for 62 million affiliates from the formal labor market (51% of the population). Figures based on databases of IMSS performance indicators, which originated from electronic health records at primary care services and hospital-based registries.

Afrobarometer surveys (AFRO) http://www.afrobarometer

.org/data

Algeria, Benin, Botswana, Burkina Faso, Burundi, Cameroon, Cape Verde, Cote d’Ivoire, Egypt, Ghana, Guinea, Kenya, Lesotho, Liberia, Madagascar, Malawi, Mali, Mauritius, Morocco, Mozambique, Namibia, Niger, Nigeria, Senegal, Sierra Leone, South Africa, Sudan, Swaziland, Tanzania, Togo, Tunisia, Uganda, Zambia, Zimbabwe

(All countries surveyed between 2011-2013)

District Level

Household and

Facility Survey

(DLHS)

http://rchiips.org/ India, 21 states

2012-2014

WHO Stepwise

Approach to

Surveillance

(STEPS)

http://www.who.int/ncds/su rveillance/steps/en/

Bhutan 2014, Burkina Faso 2013, Guyana 2016, Kenya 2015, Saint Vincent & The Grenadines 2013, Swaziland 2014, Uganda 2014

3. HQSS Commission-led Survey of User Experience in 12 LMICs

Study aims

We commissioned an internet survey of user experience in 12 LMICs from the RIWI

Corporation. The survey included questions about demographics, healthcare utilization, the respondent’s last healthcare experience, their perceptions of the health system and the healthcare vignettes designed to elicit thoughts about expectations of care.

Study design and sample

We used random domain intercept technology (RDITtm) to collect survey data through the internet. The methodology is patented by the Riwi Corporation and randomly exposes internet users to an online survey instrument. “RDIT™ delivers anonymous opt-in surveys to random web users who are surfing online by typing directly into the URL bar. When these users make data input errors by typing in websites that no longer exist, or by mistypes on non-trademarked, secure websites that RIWI owns or controls at any given time, RIWI invites these random web users, filtered through a series of proprietary algorithms.”

We used RDITtm to randomly sample web users aged 18 or older in 12 LMIC countries in

August and September of 2017. The survey closed when at least 1,000 surveys were completed per country. Countries were purposefully chosen to represent LMICs from all world regions. In order to optimize population representativeness, only countries with internet penetration rates over 20% were selected. Included countries were Senegal, Ghana, Kenya, India, Nigeria, Morocco, Indonesia, South Africa, Lebanon, China, Mexico, and Argentina. The survey instrument was translated to country-appropriate languages and back-translated by native speakers to check for accuracy. Each respondent received a maximum of 23 questions (exact number was dependent on response content and skip-logic).

Sampling weights

Sampling weights were created post-stratification using a ranking algorithm to approximate a nationally representative sample based on the respondent’s age, gender, urban/rural and education. The age and gender weight targets were created based on the Census Bureau's 2017

Population Estimates of the country's population. The urban and rural targets were created based on the Central Intelligence Agency database. The education targets for most countries were created based on educational attainment data from the UNESCO Institute for

Statistics. Education targets for Nigeria were created based on data from the Demographic and

Health Survey conducted by the National Population Commission of Nigeria and education targets for Morocco were based on educational enrolment data from the High Commission for the Plan of Morocco.

Ethical approval

This study (protocol number IRB17-0907) was reviewed by the Harvard University Human

Research Protection Program and deemed exempt from full review.

source:%20https://www.census.gov/population/international/data/idb/informationGateway.php.

source:%20https://www.census.gov/population/international/data/idb/informationGateway.php.

4. Definitions of quality measures included in section 2

Definitions for all the quality measures calculated by the Commission are included below according to the order they are presented in Section 2.

Figure 2A in the report presents composite measures of quality based on adherence to evidence-based guidelines for maternal and child health services. Using guidelines from the WHO, we identified essential elements of care for family planning, antenatal care and sick child care consultations.1-4 We matched these to indicators available in the Service Provision Assessment

Surveys and the baseline surveys of Results Based Financing impact evaluations. Labor and delivery care quality was assessed using a previously validated index.3 The quality indices were calculated as the percentage of items fulfilled per visit, to provide a continuous quality score scaled from 0 to 100. The table below describes the individual items included in the composite quality indices.

Service Index components: Recommended care items

Family planning Asked: age, current breastfeeding, # living children, chronic illness history, reproductive intentions, last delivery, last period, menstruation regularity, smoking history, STI symptoms, desired timing

Exam: blood pressure, pelvic exam, weight

Prescribed at least one family planning method

Counseling: explained how to use method, possible side effects and what to do if have problems, discussed partner status and attitude, risk of STI/HIV, condoms, dual method use, asked about concerns, discussed follow-up visit

Privacy: ensured visual and auditory privacy, assured client confidentiality

Communication: Used visual aids, checked and wrote on card

Antenatal care Asked: danger signs, pregnancy history, last period, previous complicationsa

Exam: weight, fundal height, oedema, vaginal exam, blood pressure, ultrasound, fetal heart ratea

Test: syphilis, HIV, anemia, blood group, urine

Prevention: tetanus toxoid injection, iron/folic acid, IPTp against malariaa

Counseling: nutrition, sleeping under ITN, birth plan, supplies for home delivery, breastfeedinga, postpartum/postnatal carea, pregnancy spacinga, wrote on card

Labor and delivery care

Assessment and exam: HIV status, headaches or blurred vision, vaginal bleeding, blood pressure, pulse, washes hands, wears gloves

First stage of labor: explains what will happen during labor, prepares uterotonic drug, uses partograph, prepares bags for neonatal resuscitation

Second and third stages of labor: correctly administers uterotonic, assess completeness of placenta and membranes, perineal and vaginal lacerations

Immediate newborn and postpartum care: dries baby, skin-to-skin, ties or clamps cord, takes mother’s vital signs, palpates uterus, initiate breastfeeding

Sick child care Asked: ability to drink, normal and sick feeding patternb, cough or difficulty breathing, diarrhea, fever, vomiting, convulsions, maternal HIV status, ear problems

Exam: weight, plots weight on chart, temperature, pallor, oedema, MUAC, jaundice, count respirations, check mouthb

Gives or checks card for: immunizations, vitamin A, deworming

Counseling: how to administer meds if prescribed, gives directions for feeding, danger signs, scheduled or discussed return visit, gave diagnosis, keeping infant warmb a Follow-up antenatal care visits only b For children under 2 months old only

All other quality indicators presented in section 2 are defined here.

Figure number or report section Variable name Data source Data type Numerator Denominator

Patient assessments

Postpartum checkup for mothers in a health facility after delivery and before discharge

DHS, MICS Self-report Women who were examined or asked questions about their health

Women who delivered in a health facility

Patient assessments

Blood pressure, urine and blood samples taken during ANC with skilled provider

DHS, MICS Self-report

Women who received all of the following at any point during their last pregnancy: blood pressure checked, urine and blood samples taken

Women who had at least one antenatal care visit with a skilled provider during their last pregnancy (doctor, nurse or midwife or country-specific skilled provider)

Figure 2B Diagnostic accuracy, Malaria with anemia SDI Provider test

Correctly identified diagnosis as malaria with anemia from vignette All providers

Figure 2B Diagnostic accuracy, Neonatal asphyxia SDI Provider test

Correctly identified diagnosis as neonatal asphyxia from vignette All providers

Figure 2B Diagnostic accuracy, Diarrhea SDI Provider test Correctly identified diagnosis as diarrhea from vignette All providers

Figure 2B Diagnostic accuracy, Pneumonia SDI Provider test Correctly identified diagnosis as pneumonia from vignette All providers

Figure 2B Diagnostic accuracy, Diabetes SDI Provider test Correctly identified diagnosis as diabetes from vignette All providers

Figure 2B Diagnostic accuracy, Postpartum hemorrhage SDI Provider test

Correctly identified diagnosis as postpartum hemorrhage from vignette

All providers

Figure 2B Diagnostic accuracy, Tuberculosis SDI Provider test Correctly identified diagnosis as TB from vignette All providers

Figure 3 and figure 11A and B

Received appropriate number of tetanus vaccinations and iron supplements among women who attended ANC

DHS, MICS Self-report

Women who received an appropriate number of tetanus injections (defined as either 2 during current pregnancy or 1 during current pregnancy and 1 in the past 3 years) and were given or bought iron supplements.

Women who had at least one antenatal care visit with a skilled provider during their last pregnancy (doctor, nurse or midwife or country-specific skilled provider)

Figure 3 and figure 11A and B

Received antibiotics when seeking care at a facility for symptoms of pneumonia

DHS, MICS Self-report Children who received antibiotic pills, syrup or injections

Children under 5 who, in the past 2 weeks, have suffered from symptoms consistent with pneumonia (a cough accompanied by short, rapid breathing and difficulty breathing as a result of a problem in the chest) and were taken to a medical facility for treatment (including public sector and medical private sector facilities, except for pharmacies)

Figure 3 and figure 11A and B

Received oral rehydration therapy when seeking care at a facility for diarrhea

DHS, MICS Self-report

Children who received oral rehydration therapy (from oral rehydration salts (ORS), pre-packaged ORS liquid or other homemade fluids)

Children under 5 who had diarrhea in past 2 weeks and were taken to a medical facility for treatment

(including public sector and medical private sector facilities, except for pharmacies)

Figure 3 Percent of people aware of their

HIV status who are on ART UNAIDs

Global estimate People on ART

People living with HIV who know their status

Figure 3

Received minimally adequate treatment among those diagnosed with MDD

World Mental

Health Survey5 Self-report

Received minimally adequate treatment: either pharmacotherapy

(at least 1 month of a medication, plus at least 4 visits to any type of medical doctor) or psychotherapy

(at least 8 visits with any professional including religious or spiritual advisor, social worker or counsellor)

Diagnosed with major depressive disorder in the last 12 months

Counseling

Told about potential side effects when first prescribed contraceptives

DHS, MICS Self-report

Women told about side effects or problems the respondent might have when first obtained the method she is currently using.

Women who were prescribed a family planning method from a health facility

Counseling

Told about danger signs or where to go in case of complications during ANC with skilled provider

DHS, MICS Self-report

Women told about potential danger signs to look for during pregnancy or where to go in case of complications

Women who had at least one antenatal care visit with a skilled provider during their last pregnancy (doctor, nurse or midwife or country-specific skilled provider)

Counseling

Counseled on HIV (transmission, prevention, getting tested) during

ANC with skilled provider

DHS, MICS Self-report

Women told about AIDS transmitted from mother to child or things to do to prevent AIDS or getting tested for AIDS virus.

Women who had at least one antenatal care visit with a skilled provider during their last pregnancy (doctor, nurse or midwife or country-specific skilled provider)

Counseling Stated the child diagnosis during sick child consultation SPA, RBF Observations

Observed stating the diagnosis during the consultation Sick child care observations

Counseling

Was told to quit smoking or not start by a healthcare provider among those diagnosed with chronic condition in past 12 months

STEPS Self-report Healthcare provider told quit or not start smoking

People diagnosed by a physician or other health care provider with high blood glucose, high blood pressure, or high blood cholesterol in the past 12 months

Counseling

Was told to maintain healthy body weight or lose weight by a healthcare provider among those diagnosed with chronic condition in past 12 months

STEPS Self-report Healthcare provider told to maintain health body weight or lose weight

People diagnosed by a physician or other health care provider with high blood glucose, high blood pressure, or high blood cholesterol in the past 12 months

Counseling

Was told to start or do more physical activity by a healthcare provider among those diagnosed with chronic condition in past 12 months

STEPS Self-report Healthcare provider told to start or do more physical activity

People diagnosed by a physician or other health care provider with high blood glucose, high blood pressure, or high blood cholesterol in the past 12 months

Counseling

Was told about dietary change by a healthcare provider among those diagnosed with chronic condition in past 12 months

STEPS Self-report

Healthcare provider told to reduce salt, eat ≥5 servings of fruit and vegetables a day, or reduce fat in diet.

People diagnosed by a physician or other health care provider with high blood glucose, high blood pressure, or high blood cholesterol in the past 12 months

Counseling Doctor gave advice on diet and exercise IDB Self-report

Doctor gave advice on exercise, diet and stress Has at least one chronic condition

Prevention, detection

Proportion up-to-date with preventive exams IDB Self-report

Had blood pressure checked in last year and cholesterol measured in last 5 years

All adults over 18 years old

Prevention, detection

Women aged 50-69 who had a mammogram in past 3 years SAGE Self-report

Had a mammogram in the past three years Women age 50-69 years old

Prevention, detection Tested for HIV during ANC DHS, MICS Self-report Women tested for HIV

Women who had at least one antenatal care visit

Continuity, Figure 11A and

B

Retained in ANC until the 4th visit among women who had at least 1 visit

DHS, MICS Self-report

Women who had at least 4 antenatal care visits during their last pregnancy

Women who had at least one antenatal care visit during their last pregnancy

Continuity, Figure 11A and

B

Received the third dose for DTP by one year of age, among those who had received the 1st dose

DHS, MICS Self-report

Children who had all three doses of the diphtheria, tetanus, pertussis vaccine by one year of age

Children who received at least one dose of the DTP vaccine

Continuity

Percentage of people living with

HIV known to be on ART 12 months after starting

DHS, MICS

Global estimate

On ART 12 months after beginning

ART

People living with HIV on ART

Timely care

Woman was checked within 1 hour of delivery after giving birth in a facility

DHS, MICS Self-report

Women who were examined or asked questions about their health within one hour of delivery

Women who delivered in a health facility

Timely care Timely breast cancer diagnosis IMSS Electronic health records

Number of women with opportunity for timely breast cancer diagnosis, defined as women with time between mammography and histopathological diagnosis less than

30 calendar days

Number of women ≥25years with breast cancer diagnosis who had mammography and histopathological confirmation in

Timely care Timely cervical cancer diagnosis IMSS Electronic health records

Number of women with opportunity for timely cervical and uterus cancer diagnosis, defined as women with time between Papanicolaou and histopathological diagnosis less than

30 calendar days

Number of women ≥25years with cervical and uterus cancer diagnosis who had Papanicolaou and histopathological confirmation in 2016

Timely care Timely breast cancer treatment IMSS Electronic health records

Number of women with opportunity for timely breast cancer treatment, defined as women with time between histopathological confirmation of Breast Cancer diagnosis and the beginning of therapy less than 21 days

Number of women ≥25years with breast cancer diagnosis who had mammography and histopathological confirmation in

Timely care Timely cervical cancer treatment IMSS Electronic health records

Number of women with opportunity for timely cervical and uterus cancer treatment, defined as women with time between histopathological confirmation of cervical and uterus cancer diagnosis and the beginning of therapy less than 21 days

Number of women ≥25years with cervical and uterus cancer diagnosis who had Papanicolaou and histopathological confirmation in 2016

Figure 4

Never experienced lack of attention or respect from public facility staff

AFRO Self-report

Never experience problem with lack of attention or respect from staff at local public clinic or hospital in the last 12 months

Visited public health facility in past 12 months

Figure 4 Rated respect at last outpatient visit as good or better HQSS Self-report

Rates quality of respect the provider showed at their last visit as "Good", "Very good" or "Excellent"

Visited any health facility in past

12 months

Figure 4 Regular GP explains things in a way that is easy to understand IDB, CWF Self-report

Regular doctor explains things in a way that is easy to understand always or often

Has a regular doctor

Figure 4 Rated how the provider listened at HQSS Self-report Rates how well the provider listened Visited any health facility in past last outpatient visit as good or better during their last visit as "Good", "Very good" or "Excellent"

12 months

Figure 4 Regular GP spends enough time with you IDB, CWF Self-report

Regular doctor spends enough time with you always or often Has a regular doctor

Figure 4

Rated how much time the provider spent with patient at last outpatient visit as good or better

HQSS Self-report

Rates the time the provider spent with him/her in the past visit as

"Good", "Very Good" or

"Excellent"

Visited any health facility in past

12 months

Figure 4 Rated wait time at last outpatient visit as good or better HQSS Self-report

Rated the wait time at their last facility visit as "Good", "Very good" or "Excellent"

Visited any health facility in past

12 months

Figure 4 Did not have a problem with the wait time at this visit SPA Self-report

Does not have a problem with the amount of time s/he waited before being seen at the visit today

All exit interview respondents

Figure 4

Did not have a problem with the amount of explanation received during this visit

SPA Self-report

Does not have a problem with the amount of explanation received during this visit

All exit interview respondents

Morbidity

Number of women who reported ever experiencing symptoms of a fistula among those whose last birth was attended by a skilled provider per 1,000

DHS Self-report

Ever experienced a constant leakage of urine or stool, occurred after a delivery or some other event.

Most recent delivery was with a skilled birth attendant

Morbidity Percent of people on ART who achieve viral suppression UNAIDs

Global estimate

Achieve viral suppression (less than

1000 copies per ml) People living with HIV on ART

Morbidity

Percent of people receiving treatments for chronic lung disease who report no symptoms from the disease in the past 2 weeks

SAGE Self-report

Experienced no symptoms

(shortness of breath at rest, coughing or wheezing for 10 minutes or more or coughing up sputum or phlegm for most days of the month) in the last two weeks

Taken any medications or other treatment (like oxygen) for diagnosed chronic lung disease in the past 12 months

Morbidity

Percent of people receiving treatments for depression who report no symptoms from the disease in the past 2 weeks

SAGE Self-report

Experienced no symptoms in past twelve months, following definition of Arokiasamy et al, shown below*.

Taken any medication or other treatment (including attending therapy or counseling sessions) for diagnosed depression in the past 12 months

Figure 7

Believes the system works pretty well and only minor changes are needed

HQSS, IDB,

CWF

Self-report

Agrees with statement: On the whole, the system works pretty well and only minor changes are necessary to make it work better.

All respondents

Figure 7 Thinks government handles AFRO Self-report Says government is doing very well All respondents improving basic health services well at improving basic health services.

Figure 7 Confident that if sick will receive the most effective treatment IDB Self-report

Confident that if sick will receive the most effective treatment All respondents

Figure 7

Confident that if got sick tomorrow, could get the care s/he needed

HQSS Self-report

"Somewhat confident" or "Very confident" would be able to receive effective treatment from the health system if s/he becomes very sick tomorrow

All respondents

Utilization and bypassing, Figure 8

Bypass public facilities DLHS Self-report

Usual source of treatment when household member gets sick is private facility, NGO, home treatment, chemist, or non-medical shop

All respondents

Utilization and bypassing, Figure8

Bypassers cited at least one quality concern DLHS Self-report

Selected any of the following options for not attending a government facility: poor quality of care, doctor not available, health personnel often absent, no adequate infrastructure, drugs not available, distrust for government facilities, waiting time too long, or facility timing not convenient.

Households that do not use government facilities for their main source of treatment when a member gets sick

Panel 3 Very satisfied with family planning visit SPA Self-report

Reported "very satisfied" with today's visit Family planning clients

Panel 3 Very satisfied with ANC visit SPA Self-report Reported "very satisfied" with today's visit ANC clients

Panel 3 Very satisfied with sick child visit SPA Self-report Reported "very satisfied" with today's visit Sick child caregivers

Panel 3

Rated hypertension patient vignette as good, very good or excellent quality

HQSS Self-report

Reported vignette quality was

"good", "very good" or "excellent".

Vignette: [Anthony] is a 45-year old man with high blood pressure who needs a regular check up. As the health facility, the nurse does greet him, introduce herself and change his medications. She does not ask about his symptoms or check his blood pressure.

All respondents who completed the survey through question 20.

Figure 9 Reports assistance from PCP in IDB Self-report Regular doctor helps to coordinate Has a regular doctor coordinating care among those who have a regular PCP care

Figure 9

Primary care provider (PCP) knows medical history among those who have a PCP

IDB Self-report Regular provider knows medical history Currently has a regular doctor

Figure 9

Used emergency room for a condition that could have been treated at the primary care level

IDB Self-report Used emergency room for a primary health care treatable case

At least one emergency room visit in the past 2 years

Figure 9

Has a national universal access phone number for pre-hospital care

(yes/no)

WHO Global

Health

Observatory

Global estimate

Existence of a universal access telephone number for pre-hospital care

Low and middle income countries

Figure 9

Is able to transport at least 50% of patients seriously injured by ambulance following road traffic crashes (yes/no)

WHO Global

Health

Observatory

Global estimate

Reported % of seriously injured patients transported by ambulance Low and middle income countries

Figure 9

Specialist had basic medical information about the patient from the GP

IDB Self-report Specialist had basic information from regular doctor

Seen any specialist doctors in the past 2 years

Figure 9 GP subsequently had the results after the specialist visit IDB Self-report

Regular doctor had up-to-date information after specialist visit

Seen any specialist doctors in the past 2 years

Figure 11C Rated overall quality of visit as good or better HQSS Self-report

Rates the overall quality of the past visit as "Good", "Very Good" or

"Excellent"

Visited any health facility in past

12 months

Figure 11C Rated the provider's knowledge at visit as good or better HQSS Self-report

Rates the time the provider's knowledge during past visit as

"Good", "Very Good" or

"Excellent"

Visited any health facility in past

12 months

Section 2 – Who receives worse quality care?

Treated poorly by health staff because of identity HQSS Self-report

Reported health staff had ever treated them poorly for one of the following reasons: poverty, religion, ethnicity or language, immigration or migrant status, education, romantic or sexual attraction to someone of the same sex, type of illness, gender, other

All respondents

* The numerator for the indicator Percent of people receiving treatments for depression who report no symptoms from the disease in the past 2 weeks calculated using SAGE data is based on the definition of depression by Arokiasamy et al6, copied below.

Question No Question text and algorithm to ascertain disease

During the last 12 months, have you had a period lasting several days when you felt sad, empty or depressed?

During the last 12 months, have you had a period lasting several days when you lost interest in most things you usually enjoy such as personal relationships, work or hobbies/recreation?

During the last 12 months, have you had a period lasting several days when you have been feeling your energy decreased or that you are tired all the time?

If any of the above three questions are yes then following set of questions were asked

4 Was this period [of sadness/loss of interest/low energy] for more than 2 weeks?

5 Was this period [of sadness/loss of interest/low energy] most of the day, nearly every day?

6 During this period, did you lose your appetite?

7 Did you notice any slowing down in your thinking?

8 Did you notice any problems falling asleep?

9 Did you notice any problems waking up too early?

During this period, did you have any difficulties concentrating; for example, listening to others, working, watching TV, listening to the radio?

11 Did you notice any slowing down in your moving around?

12 During this period, did you feel anxious and worried most days?

During this period, were you so restless or jittery nearly every day that you paced up and down and couldn’t sit still?

14 During this period, did you feel negative about yourself or like you had lost confidence?

15 Did you frequently feel hopeless - that there was no way to improve things?

16 During this period, did your interest in sex decrease?

17 Did you think of death, or wish you were dead?

18 During this period, did you ever try to end your life?

Algorithm

To ascertain the depression from this set of questions two set of variables were computed. First set was based on the questions1, 2, 3, 4, 5 and 16. From this set three variables were computed taking values 0 and

1: a) first variable takes value 1 if response to any of questions 1, 4, and 5 was yes. b) second variable takes value 1 if question 2 or 16 has response yes. c) the third variable takes value 1 if question 3 has response yes. The second set of variables was based on questions 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17 and

18. From these questions seven variables were computed. a) first variable takes value 1 if response to questions 14 or 15 is yes. b) second variable takes value 1 if response to questions 12 or 13 is yes. c) third variable takes value 1 if questions 17 or 18 has response yes. d) fourth variable takes value 1 if questions 7 or 10 has response yes. e) fifth variable takes value 1 if response to questions 11 is yes. f) sixth variable takes value 1 if response to questions 8 or 9 is yes. g) seventh variable takes value 1 if the response to question 6 is yes. These newly created variables from the respective sets were added to obtain two new variables: first consisting sum of first set of variables (maximum value 3) and second consisting sum of second set of variables (maximum value 7). Based on these two variables, a respondent is said to suffer from depression if he has value for the first variable to be 2-plus and the value for second variable to be 4-plus.

5. Panel 2: Beyond the numbers – experiences in the health system: review

Between November 1st and November 25th, 2017, we conducted a rapid review of qualitative studies on the experiences of the poor within the health system in LMICs. A search of PubMed returned 335 results, which were screened using titles/abstracts. Of these, 60 were selected for full-text review and 39 were selected for final inclusion. The reference lists of included studies were reviewed for relevant papers, yielding an additional 9 studies. Themes and quotes were extracted with the focus on explaining how poor people experience elements of competent care and user experience in the health system.

Search strategy:

[“healthcare disparities” or “differential treatment”] + [“disparities” or “inequality” or

“inequalities”] + [“poor” or “marginalized” or “indigent” or “disadvantaged” or “low-income” or

“vulnerable”] + [“informant” or “focus group” or “qualitative”] + [LMIC hedge]

6. Adjusting for quality in the Lives Saved Tool

Victoria B. Chou, Mufaro Kanyangarara, and Neff Walker

Full manuscript forthcoming

Using a linking approach to combine available health facility and household survey data, baseline levels of coverage for key maternal and neonatal interventions were estimated in 17 countries. The maternal and newborn health interventions (n=19) span across the continuum of care covering the antenatal to postnatal period. The impact of scaling up these specific services was quantified using the Lives Saved Tool (LiST) for a representative sample of 81 focus countries adopted by the Countdown to 2030 consortium. Of the global burden, these countries represent an estimated 89% of the neonatal deaths, 96% of the maternal deaths, and 87% of the stillbirths occurring worldwide.

Baseline coverage for countries with available data to analyze a linked dataset was estimated based upon uniform indicator definitions specifying the minimum components required for adequate delivery of each individual maternal or neonatal intervention. Median values derived from this sample were applied as the baseline or starting coverage for countries without a linked dataset and reported levels of utilization (e.g. proportion of women who attended four or more antenatal visits (ANC4+) or percentage of births delivered in a health facility) were applied as the country-specific targets or “cap” for modeling improvements in the quality of care provided.

Ranges were calculated by applying lower or upper bounds to the estimates for baseline coverage in the set of 81 modeled countries.

7. Why quality of maternal mental health care may suffer for vulnerable groups: An illustration with perinatal depression care in primary care setting in Nigeria (Panel

4)

Olatunde Olayinka and Oye Gureje

We conducted a formative study as a part of an implementation research project to assess the factors that may promote or hinder the delivery of quality service to women with perinatal depression. The project, Scaling up Care for Perinatal Depression for Improved Maternal and

Infant Health (SPECTRA), is being conducted in randomly selected primary health care (PHC) facilities across all the 11 local government areas (LGAs) in and around Ibadan city in the south-western part of Nigeria.

The formative study sought to address three questions: 1) What is the current organizational structure of the clinics in regard to the delivery of quality chronic care? 2) How able are providers in these clinics to identify and provide treatment for women with perinatal depression?

And 3) How do women with maternal depression rate the standard of care in the clinics?

To address the first question, we administered the Assessment of Chronic Illness Care (ACIC) (8) to the facility managers in 23 maternal care clinics. The ACIC is a widely used tool to evaluate the standard of care provided to patients who are in need of sustained care. Specifically, we assessed the capacity of the maternal care facilities to provide care and support to patients with perinatal depression. The items of the tool, which are organised into 7 different domains, are scored on a Likert scale, giving an average domain score that ranges from 0 to 11. A score between "0" and "2" corresponds to the lowest level of support for chronic care while a score between "9" and "11" indicate fully developed chronic illness care programme.

To address the second question, consecutively registered women presenting for routine antenatal care and who had previously consented to participate in the study were screened with Edinburgh

Postnatal Depression Scale (EPDS) (9) to identify those with perinatal depression, after they had been seen by the primary health care workers (PHCWs). The EPDS has been validated in our setting and, for the purpose of this exercise, a score of 10 or more out of a possible total of 30 was regarded as indicative of moderate depression. Over a period of 6 months, 218 of such cases were identified. We made a determination of whether a woman who screened positive to depression using the EPDS had been recognised as having a psychological problem by the maternal care provider. For this, we used a fairly broad definition of detection. First, we reviewed the case records made on each of the patients by the providers to see whether any entry had been made to indicate that any psychological problem had been noted or treated. That is, the entry did not have to specify a diagnosis, just an indication of any symptoms or reported emotional problem. Second, we conducted an assessment of the women who screened positive in their homes within 72 hours of being screened in the maternal care clinic. We enquired from them whether they had been asked any question about their mood, sleep, worry or any emotional concern. Irrespective of the answer they provided and to address the third research question, the respondents were asked to rate 1) the overall quality of care provided in the clinics and 2) their level of satisfaction with care provided. Each of the items was framed in such a way that it can be rated on a binary “Yes/No” scale.

Section 3: The ethical basis of high-quality health systems

1. Social accountability for health quality review

We conducted a scoping review of systematic reviews on community accountability mechanisms in the health sector in low-and-middle income countries. We searched PubMed, Embase and

Web of Science using a combination of search terms related to community accountability mechanisms. We excluded non-English language reviews, reviews published before 2000, reviews on high income countries, and reviews on accountability mechanism outside of the health sector. We assessed each study for quality, using GRADE ratings for quantitative studies and GRADE-CERQual for qualitative studies and excluded any studies rated as “Low” or “Very

Low” quality.

2. Actions to support legal and social accountability, review (Panel 7)

Between September 18th and November 10th, 2017, we conducted a synthesis review informed by realist principles of incorporating iteration and purposeful article selection based on relevance to the research question. A WHO expert group with representatives from reproductive, maternal and child health and law and governance identified the underlying assumptions to the accountability-quality interface and designed a conceptual framework. We conducted PubMed and Google Scholar searches using the search terms “accountability,” “quality” and “health” for

English language reviews, systematic reviews or reports in low-income and middle-income countries (World Bank classification) since 2000. We combined this with suggestions of relevant literature from the expert group and snowballing using the reference lists. Articles were included based on their ability to provide information relevant to our research question and aim. Data from the identified articles was extracted based on a pre-defined data extraction table which sought to explore elements of what accountability approaches work, for whom, and why and in what contexts. This focused on explaining the relationships between accountability and quality, the contexts in which they interact, the mechanisms by which accountability for quality in health work and the quality outcomes described in the secondary literature. Data extraction was conducted by two reviewers, with synthesis of results in real time in discussion with the expert group.

Section 4: Measuring health system quality

1. Quality subdomains in global, cross-national, and national measurement sets, indicator review (Figure 12)

Hannah H. Leslie, Naima T. Joseph, David Sando, Carmen Santamaría, Laura del Pilar Torres-

Arreola, Svetlana V. Doubova

Manuscript with detailed results of Mexican indicator review forthcoming

We purposively identified indicator sets defined by global and multi-country organizations for health system assessment or global development. We additionally obtained the most common standard health facility assessments and the household survey used most broadly in low- and middle-income countries. We invited national commissions to provide lists of national health system indicators and were able to find several such lists publicly available from Ministries of

Health or similar entities.

The reviewed sets were:

• Countdown 2030 Indicators approved for 2017, Tier 1 and Tier 2:

http://countdown2030.org/wp-content/uploads/2017/12/Technical-Review-

Process_tables.pdf

• EURO Health for All Database, July 2016 version: http://data.euro.who.int/hfadb/

• OECD Health Indicators as of September 2017 in categories: health care quality, health expenditure and financing, health status, health care resources, health care utilisation, social protection, non-medical determinants of health.

http://stats.oecd.org/index.aspx?datasetcode=Health

• SDG health indicators: indicators in the July 6 2017 Resolution adopted by the General

Assembly on Work of the Statistical Commission pertaining to the 2030 Agenda for

Sustainable Development (A/RES/71/313), Annex

(https://unstats.un.org/sdgs/indicators/Global%20Indicator%20Framework_A.RES.71.31

3%20Annex.pdf) and indicators within the proposed UHC coverage index for SDG 3.8.1:

http://www.thelancet…