Attachment J.5-Targeting Malaria Mortality - Prioritizing Districts for CCM Malaria in Uganda - 0629.pdf

JUNE 2020

Prepared by GiveWell for the

U.S.’s President’s Malaria Initiative (PMI) /

Uganda

Targeting Malaria Mortality

Prioritizing Districts for Community Case Management of Malaria in Uganda

Report © 2020 GiveWell

Cover Photo Credit: “UGANDA – MCSP” by US Agency for International Development is licensed under CC BY-NC 2.0 https://www.flickr.com/photos/usaid_images/49784864456/in/photolist-2iRjrJ1-7XA7iP-4BQX4Q-7d4s3R-dCFZVw-dCFZeN-g7LyDX-ak5d2V-7fTEDG-ks8jSH-84VSzG-84STz4-84W7ks-84SHRz-84W3j3-a1rmh3-a1rmhd-a1rrxw-a1rmh7-552ayH-ks8k1D-a1rrxm-a1rrxJ-a1rrxN-a1rrxC-dCAywp-7d4rWR-7XA6hV-dCAz72-8rbVaq-8rmXYp-8rbyZL-7XDjQu-7XA5ai-dCFZ4f-7d8j8G-7XA5uZ-7XDiBG-7XDiw3-7XDk3Q-7XA5j6-7XA53r-a1oEU6-84SFSV-8r8Njr-8rbVs9-8rbVd1-8rbVJJ-8r8qx6-8rbyth https://creativecommons.org/licenses/by-nc/2.0/

Contents Executive Summary .................................................................................................... i

Introduction

Prioritization model and framework

Quantitative model

Malaria burden (column D)

Cost (column E)

General reduction in malaria mortality due to CCM (column G)

Adjustment to mortality reduction for ACT stockouts (column H)

Adjustment to mortality reduction for access to health facilities (column I)

Results

Qualitative factors

Limitations

Comparing approaches to prioritizing districts

Endnotes

One

Two

Three

Four

Five i

Executive Summary The U.S.’s President’s Malaria Initiative (PMI) plays an essential role in delivering cost-effective, life-saving malaria interventions in 27 focus countries. Yet with limited resources, difficult decisions must be made as to how to prioritize interventions and geographies within each country. In recognition of this, PMI’s most recent strategy calls for flexible approaches tailored to local settings as well as appropriate analysis of which mix of interventions provide best value for money.1 Key partners, such as the World

Health Organization and Uganda’s

National Malaria Control Division emphasize the importance of targeted approaches based on local data tailored to their unique settings.2

Building on these shared interests and

PMI’s track record of investing in proven, cost-effective interventions, GiveWell created a cost-effectiveness model to support the prioritization of districts for

Community Case Management (CCM) in

Uganda.

GiveWell aimed to identify which districts would have the lowest cost per malaria death averted due to CCM. The key factors in this model are:

● Malaria burden,

● All-in cost per person reached

● Access to health facilities, and

● Availability of key treatments such as ACTs.

We considered a variety of factors that we believe are effectively reflected in the above (such as coverage of existing malaria-related interventions) or that were added as qualitative factors that can be considered along the quantitative results (such as poverty rates by district and coverage of Uganda’s community health workers, called

Village Health Teams).

1 PMI, “President’s Malaria Initiative Strategy 2015–2020,” p. 18 2 The Republic of Uganda, Ministry of Health, “The Uganda Malaria Reduction and Elimination Strategic Plan 2021-2025,” (March 2020), unpublished draft, p. 35 & 62.; World Health Organization, High Burden to High Impact, p. 4.

Prioritization of CCM Malaria in Uganda

“Targeting the most cost-effective districts is estimated to save more than twice as many lives as alternative approaches to targeting, at no additional cost” https://www.pmi.gov/docs/default-source/default-document-library/pmi-reports/pmi_strategy_2015-2020.pdf?sfvrsn=24 https://apps.who.int/iris/bitstream/handle/10665/275868/WHO-CDS-GMP-2018.25-eng.pdf?ua=1 ii

We find substantial district-level variation in cost-effectiveness. The most cost-effective districts appear to be more than ten times more cost-effective than the least cost-effective districts. Figure 1 maps these results for all districts in Uganda, excluding the districts which are supported by the Global Fund.

Targeting the most cost-effective districts is estimated to save more than twice as many lives as alternative approaches to targeting, at no additional cost. The chart at the right shows how targeting the most cost-effective districts differs from alternative approaches

(e.g. targeting the highest burden districts).

The proposed approach saves almost 7 times more lives compared to a non-targeted approach.

Limitations

Like all models, this analysis is a simplified representation of the world. Additional factors will be important for PMI to reach a final decision about which districts to support with CCM Malaria. These may include:

● Qualitative favors contained in the results spreadsheet, but which do not directly affect the prioritization of districts (e.g. poverty rates and other interventions like IRS)

● The presence of other funders already supporting CCM in relevant districts. We exclude districts proposed for Global Fund funding from our primary table. Other partners also support CCM and these are included in the results table.

● Context-specific information on particular districts which could affect prioritization, such as the quality of existing Village Health Team networks.

Our detailed model and qualitative framework are available online.

3 World Health Organization, “WHO Malaria Policy Advisory Committee (MPAC) meeting report,” (October 2019), “There is broad recognition that in many cases, the resources available are not sufficient to fund all of the elements of these national strategic plans. In this situation, countries must make difficult choices, trying to make the best use of the limited resources available.”

World Health Organization3 https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=1830312649

Introduction U.S.’s President’s Malaria Initiative plays an essential role in delivering cost-effective, life-saving malaria interventions. Yet with limited resources, difficult decisions must be made as to how to prioritize different interventions and geographies within each country. In recognition of this, PMI’s most recent strategy calls for flexible approaches tailored to local settings as well as “appropriate analysis of which mix of interventions provide best value for money.”

Indeed, the interest in ensuring limited resources are targeted to where they can do the most good are shared by many partners in the fight against malaria. WHO’s High Burden to High Impact approach emphasizes that

“through better analysis and the strategic use of quality data, countries can pinpoint where to deploy the most effective malaria control tools for maximum impact.” Similarly, Uganda’s National Malaria Control Division’s

2021-2025 Draft Strategic Plan calls for the “appropriate targeting of interventions” and malaria programming

“guided and based on robust data & evidence”.

Building on these shared interests and PMI’s track record of investing in proven, cost-effective interventions, GiveWell created a cost-effectiveness model to support the prioritization of districts for Community Case

Management in Uganda.

Community Case Management of Malaria (CCM Malaria) involves the diagnosis (including the use of RDTs) and treatment of Malaria via community health workers. In Uganda, community health workers are referred to as

Village Health Teams (VHTs).

In addition to PMI’s support of basic malaria commodities, additional funding for CCM Malaria is required to support activities such as:

● Additional training to Village Health Teams,

● Providing VHTs with reporting tools

● Supporting more frequent check-ins between supervisors from the health facilities (Health Center II) and

VHTs (e.g. monthly rather than quarterly), and

● stipends to Village Health Teams and compensation for travel costs

In consultation with PMI Uganda, we created a model that prioritizes districts based on cost-effectiveness. The key inputs into the model include:

● Malaria burden

● Proxies for cost per person reached, such as population density

● Proxies for access to healthcare, such as number of health facilities per person

● Availability of key treatment commodities such as artemisinin combination therapies (ACTs)

Other factors included in our results but that do not directly affect the prioritization of districts include:

● Poverty rates

● Existing VHT capacity

● Presence of PMI-supported Indoor Residual Spraying (IRS)

Based on conversations with PMI and PMI technical guidelines, this model focuses on the impact of CCM activities on malaria specifically (rather than "integrated community case management" (iCCM), which also includes case management of pneumonia and diarrhea). We specifically focus on malaria mortality reductions among under-5 year olds, since this is the primary target population for CCM activities.

Our understanding is that the primary mechanisms through which CCM reduces deaths are: VHTs testing children for malaria, providing direct treatment with ACTs for uncomplicated malaria cases, and referring children with danger signs indicative of severe malaria to health facilities.[1]

Prioritization model and framework Our prioritization model and framework can be summarized as follows:

● We have built a simple quantitative model that uses malaria burden (drives ~65% of the quantitative conclusions), population density (~15%), access to health facilities (~15%), and ACT stockout rates (~5%) to estimate the relative cost-effectiveness of CCM by district.

● We also provide a qualitative overlay (columns O through T) of factors that seem important to prioritization but that seemed less appropriate to put into a quantitative framework (further explanation below). These include: presence of other funders (e.g. UNICEF) already supporting CCM, poverty rates, presence of active Village Health Teams, and district-level performance on coverage for other malaria-related interventions.

● To see the rough ranking of districts by cost-effectiveness, see column N.

Further details on the model and data sources below.

Quantitative model

See the complete cost-effectiveness model in sheet "Core Model".

An overview of the key variables driving the results of the cost-effectiveness model are:

● Malaria burden: ranges from ~1 death per 1,000 under-5 person-years (10th percentile of data) to ~5.5 deaths per 1,000 (90th percentile). This is a range of about 5x, which drives about 55-75% of the quantitative results.

● Cost (primarily based on population density): ranges from ~$10 to ~$15. This is a range of about 1.5x, which drives about 10-15% of the quantitative results.

● Adjustment for access to health facilities: ranges from 80% (i.e., 20% discount to cost-effectiveness) to

120%. This is a range of about 1.4x, which drives about 10-15% of the quantitative results.

● Adjustment for ACT stockouts: ranges from about 80% (i.e., 20% discount) to 100%. This is a range of about 1.2x, which drives about 5% of the quantitative results.

We also discuss other variables included in the model below, describing the model column-by-column.

Malaria burden (column D) We used 2019 district-level data from the Malaria Atlas Project to estimate malaria-attributable mortality per

1,000 under-5 year olds by district.4 Malaria-attributable mortality estimates by district in Uganda range from about 1 death per 1,000 under-5 person-years (10th percentile of data) to about 5.5 deaths per 1,000 (90th percentile). We also adjusted for indirect malaria mortality.

4 Note: The Malaria Atlas Project provided us with data for overall malaria incidence by district in 2019. We used past estimates of the ratio of incidence to mortality, GBD estimates of the fraction of all malaria mortality that occurs in under- 5 year olds (estimated to be about 72%), and estimates of indirect malaria mortality to arrive at a final estimate of malaria-attributable mortality per 1,000 under-5 year olds.

https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=175224579 https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=175224579&range=O2 https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=175224579&range=O2 https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=175224579&range=O2 https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=175224579&range=N2 https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit?pli=1#gid=175224579 https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=175224579&range=D2 https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=175224579&range=D:D https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=982421216&range=B2 https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=982421216&range=I1

Our understanding is that Malaria Atlas Project (MAP) estimates are widely seen as credible; for example, MAP is the primary data source for the "Country Profile" maps included in recent World Malaria Reports.5

Malaria Atlas Project uses data on prevalence of P. falciparum parasites from data sources such as the

Demographic and Health Surveys. It uses GPS data to map this information at a 5x5 kilometer level. It combines parasite prevalence estimates with other data such as ecological factors (e.g. rainfall, temperature) and coverage of malaria interventions to predict malaria mortality in each region.[2]

We considered using district-level DHIS2 data, but the potential for undercounting of cases in the private sector and the lack of triangulation with factors such as ecological predictors of malaria burden led us to believe that

MAP data is likely of higher quality. These two data sources imply different estimates, so this choice appears to be somewhat meaningful to conclusions about how to prioritize.6

Cost (column E) To estimate the costs of CCM programs in Uganda, we:

● Reviewed independent research on the variation in costs of iCCM and CCM programs by country.7 For this context, we ended up believing a range of ~$10 to $15 per child targeted was reasonable. We chose a somewhat tighter range than the full range we saw in the literature because we only focused on a subset of drivers of variation in cost and did not want large, uncertain variation in cost estimates to drive too much of our conclusions.8

● Analyzed key drivers of variation in costs. One plausible driver that was tractable for us to include was population density (which is associated with how many children an individual health worker is able to serve). We were not able to get district-level information for Uganda on other important factors discussed in the literature, such as how many additional fixed costs are necessary to establish a robust

VHT network in each district and how efficient supply chains are in each district.

● Modeled the effect of population density on cost. The independent research on costs did not provide a function that we could use to easily map variables like population density to total cost. So, we mapped the low end of the range of costs we saw in the literature to the high end of population densities in

Uganda and vice versa. For example, if a district had 90th percentile population density in Uganda (i.e.

high density), we assumed it had a cost of about $10 per child targeted (the low end of the range from relevant literature). Median population density districts in Uganda were assumed to cost about $12.50 per child targeted and 10th percentile population density districts were assumed to cost about $15 per child targeted.

5 See here for more.

6 See Sheet "MAP and DHIS2 burden analysis" 7 Key sources we relied on include Collins et al. 2014, Chinbuah et al. 2012, Hansen et al. 2017, Biemba et al. 2019, and an International Rescue Committee Brief. We also considered a study of iCCM in Uganda, Kastening, et. al, 2013, unpublished 8 Overall, the total cost per person targeted for iCCM-like program seems to range from about $5 to $15. See sources cited in the previous footnote.

https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=175224579&range=E2 https://malariaatlas.org/2018-world-malaria-report-is-published-using-maps-produced-by-the-malaria-atlas-project/ https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=1879816201 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4267093/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3748510/pdf/tropmed-87-11.pdf https://academic.oup.com/heapol/article/32/5/676/2997443 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7118441/ https://www.rescue.org/sites/default/files/document/955/cpcmdesignedbrieffinal.pdf

We took a "total cost" perspective here rather than a "marginal cost" perspective, since total cost is typically what was estimated in independent literature.9 Total costs include the costs of basic commodities and other costs in setting up and running Village Health Team networks.

We understand PMI supports commodity and non-commodity costs through different awards. Understanding the PMI-specific marginal costs per district will be important to determining how many additional CCM districts can be funded. However, this will not affect the ranked ordering of districts in this prioritization exercise.

General reduction in malaria mortality due to CCM (column G) To estimate the overall effectiveness of CCM malaria in each district, we reviewed relevant randomized controlled trials that assessed the effect of iCCM, CCM, or home malaria management programs on child mortality.10 We found a range of estimates and, by roughly combining the estimates according to sample size, estimated a roughly 18% reduction in malaria mortality.11

However, our analysis of this parameter was cursory. We did not deeply review this parameter because by design it does not vary across districts, so it is not particularly relevant to prioritization within Uganda. However, it is relevant to the overall cost-effectiveness of CCM for malaria, so may be worth revisiting more carefully when comparing CCM to other programs.

Adjustment to mortality reduction for ACT stockouts (column H) We adjusted the effectiveness of CCM malaria in each district to account for two factors: quality of service delivery and access to care.

Service delivery quality can have an important effect on the effectiveness of CCM Malaria.

We considered multiple ways to measure the service quality of Village Health Teams, including the number of trained VHTs, VHT reporting rates, and broader measures of service quality in health, such as % of malaria cases correctly treated in health facilities & district league tables.

We chose ACT stockout rates since it was directly related to whether Village Health Teams could effectively treat malaria in their communities.

9 We expect that marginal costs of improving CCM coverage (and therefore marginal cost per death averted) may be substantially lower than the total cost, but we would still expect marginal costs to vary by district according to population density.

10 Key sources we relied on include Okwundu et al. 2013, Han Oo et al. 2019, Amouzou et al. 2014, and Nyqvist et al. 2019.

We have not yet confirmed with relevant experts that there are no other relevant studies.

11 See this spreadsheet for more details. Note that this estimate includes substantial adjustments for internal and external validity; see spreadsheet for more. Note that some studies focused on malaria mortality specifically while others considered effects on all-cause mortality from programs that also delivered treatments for diarrhea and pneumonia. In all cases, we must make uncertain assumptions to convert from all-cause mortality effects to malaria-specific effects. We also believe it is appropriate to adjust somewhat for the expectation that programs in trials may be particularly well-run and may be less likely to replicate. Ultimately, we assume that CCM for malaria would reduce malaria mortality by about 18% in the Ugandan context. Our understanding is that PMI is supporting CCM for malaria specifically, so we have focused on estimates of impact on malaria deaths.

https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=175224579&range=G2 https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=175224579&range=H2 https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD009527.pub2/full?highlightAbstract=treatment%7Cmalari%7Cmalaria%7Ccommunity%7Ccommuniti https://link.springer.com/article/10.1186/s12936-019-2900-1 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4267100/pdf/jogh-04-020411.pdf https://www.aeaweb.org/articles?id=10.1257/app.20170201 https://docs.google.com/spreadsheets/d/1HVgddYXkn0h-EG19mp3eGXVeKKYdS6I_0lBZAkU18YE/edit?usp=sharing

Using data from Uganda’s DHIS2, we calculated the percent of days in 2019 in which health facilities were stocked out of a key ACT (Artemether/ Lumefantrine 100/20mg tablets). District averages varied from 25% of days stocked out to 0% stock out. We used this to discount the effectiveness of CCM malaria by the average % of days in each district that facilities were stocked out of ACT.

Adjustment to mortality reduction for access to health facilities (column I) We also adjusted CCM Malaria effectiveness to account for differing levels of access to health facilities.

Districts which have worse access to health facilities could benefit more from CCM malaria since Village Health

Teams provide community-based treatment options without having to travel to health facilities.

We measured access to care using two indicators:

1. The percent of the population in each region reporting difficulty accessing care due to the distance from the health facility12; and

2. Health facility density, measured as the number of health facilities per 10,000 children under 5 in each district.13

We placed greater weight on health facility density since that data was more recent and available at the district level, whereas reported difficulty in accessing care was based on the 2016 DHS household survey data representative only at the regional level. Districts with the worst access to care received a 20% boost in effectiveness, whereas districts with the best access to care received a 20% discount in effectiveness.

Results See the Key Results tab in our model for key quantitative results. These results exclude districts with funding for

CCM proposed in Uganda’s draft Global Fund application. The list of Global Fund districts is listed at the bottom of the sheet for reference.

In figure 1 & 2 below, we prioritize districts for CCM by our estimates of the programs' cost per death averted14:

● Very high priority: Under $16,000 per death averted.

● High priority: Between $16,000 and $28,000 per death averted.

● Medium priority: Between $28,000 and $42,000 per death averted.

● Lower priority: Over $42,000 per death averted.

12 See data from the 2016 DHS on "Percentage of women age 15-49 who reported that they have serious problems in accessing health care for themselves when they are sick." This survey provided the percentage of women who cite distance to health facilities as a key reason that they have problems accessing care. We use that percentage as a proxy for access to care generally. Uganda Demographic and Health Survey 2016, Pg 161, Available at:

https://dhsprogram.com/pubs/pdf/FR333/FR333.pdf 13 Government of Uganda, eHMIS. Accessed Apr 13, 2020. Note that our understanding is that this data includes public facilities and some private facilities. This indicator was suggested by PMI and also used by Global Fund in its CCM prioritization analysis.

14 A number of new districts have been recently added in Uganda. We assume that each new district has the same characteristics as the district it was previously part of. New districts are not mapped but are included in the table.

https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=175224579&range=I2 https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=1382167602&range=D:D https://dhsprogram.com/pubs/pdf/FR333/FR333.pdf

Figure 1: Prioritization of CCM Malaria for all

Ugandan districts

Figure 2: Prioritization of CCM Malaria for Ugandan districts excluding those with proposed Global Fund funding

Qualitative factors

Not all relevant factors to district prioritization could be captured quantitatively.[3] Other factors could still be qualitatively important to prioritization, so we aimed to include them in the decision framework.

The qualitative factors that we have included (see columns O through U) are:

Qualitative factor

(source in footnote)

Description

Presence of other funders already supporting CCM15

Shows which districts are supported by DFID-UNICEF-MC, Global Fund, and other actors. Working in the same district as another funder might provide the opportunity to leverage their funding. On the other hand, that district may be less neglected since another funder is supporting it.

PMI-supported IRS district16

PMI asked us to include an overlay of which districts are supported by PMI for IRS. These districts may be lower priority since they already receive substantial support. On the other hand, CCM could also be considered a key part of an IRS exit strategy

15 Based on Global Fund document. See Map "ICCM Coverage in Uganda," Pg. 20, Government of Uganda, “Global Fund Malaria Funding request" draft document.

16 These districts were provided to us via email by Dr. Kassahun Belay, Senior Malaria Resident Advisor at USAID/Uganda.

https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=175224579&range=O:U https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=175224579&range=O:U https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=175224579&range=O:U

Poverty rate17 Poverty rates are probably associated with lack of access to healthcare and malaria burden, so may be included in the quantitative model to some extent.

Nevertheless, PMI may want to further adjust for poverty rates.

Percentage of households with a VHT member in the village18

Data from Uganda's most recent Malaria Indicator Survey (2018-19) is used to show which districts reportedly have presence of VHTs. We are unsure whether PMI would prefer to improve networks in low-coverage districts or provide support in districts where VHT networks are well established and therefore perhaps more efficient.

Malaria League Table, Malaria Consortium analysis19

As discussed below, we believe that coverage of existing malaria interventions is already accounted for quantitatively in our analysis via malaria burden estimates. However, it may be appropriate to qualitatively consider which districts seem to have lower quality health facility performance on malaria indicators. In Malaria Consortium's league table, lower scores indicate lower coverage of existing interventions and lower rates of testing.

Total under-5-year-old population20

PMI may want to target districts with larger overall populations. This may be factored in, to some extent, quantitatively via the cost variable (since more populous districts may also have greater population density) but may not be fully captured.

A number of other factors we believe are already implicitly factored into our model. These include:

● Existing coverage of interventions such as insecticide-treated nets and other malaria interventions

(sometimes factored in qualitatively, but not quantitatively)

○ We believe this factor is already incorporated in the baseline malaria mortality burden, so including it explicitly would adjust for it twice, which is incorrect. We checked our conclusion by looking at the methodology of the Lives Saved Tool (LiST) and more complex, dynamic models of malaria transmission such as the Imperial College Model. We believe the LiST approach is methodologically equivalent to our approach.[4] The Imperial College Model was producing erroneous results (e.g. assuming that ACT coverage rates expanding would cause higher malaria mortality--the opposite of common sense expectation), so we deprioritized it. We have not yet reviewed other models, such as the Spectrum-Malaria model, in depth.

○ In some cases, we factor in coverage of other interventions qualitatively, such as through our inclusion of Malaria Consortium's league table, in which lower scores are given to districts with e.g. lower coverage of preventive malaria treatment for pregnant women (IPTp).

17 Based on an exercise conducted by the Uganda Bureau of Statistics (UBOS) in close collaboration with UNICEF and the World Bank. See here for details.

18Source: Uganda MIS 2018-19, Pg. 59, Table 4.4. Technically this is "Percentage of households with a CHW, CMD, or VHT member in the village/community" 19 Malaria Consortium analysis of factors such as "Test positivity rate," "Percent of Negative Malaria cases treated using ACTs," "Percent of Pregnant women receiving IPTp3," and other factors. See more here.

20 Based on estimates included in Malaria Consortium report. See here.

https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=1151316793&range=B19 https://dhsprogram.com/pubs/pdf/MIS34/MIS34.pdf https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=1426745412 https://docs.google.com/spreadsheets/d/1_L5aLsnRVW8Iv2ztg0G36JNZ8FWtVjPjJNHt6hLenEk/edit#gid=1879816201&range=P:P

Other factors we would have ideally included in our model but for which we were not able to get relevant data include:

● Indicators of VHT service delivery quality such as: Attendance, training level, frequency of visits to households, and utilization (e.g. how many commodities VHTs are using).

● Historical or forward-looking costs of CCM in Uganda.

● Community acceptability of malaria interventions.

Factors that we deprioritized incorporating in our model include:

● Presence of Living Goods or BRAC community health workers. Our understanding is that this factor was a lower priority. This factor could be added if desirable.21

● Presence of artemisinin resistance. This is a complex factor that we did not prioritize for this analysis.

Limitations

Like all models, this analysis is a simplified representation of the world. Key uncertainties that could affect the prioritization of districts in our model include:

● Variation in district-level costs - We account for one factor which likely affects district-level costs

(population density), but there may be other factors that are important and which we have not been able to account for (e.g. additional recruitment/training for districts with limited VHTs or logistics costs for remote districts).

● Context-specific information in particular districts which could affect prioritization, such as the quality of existing Village Health Team networks, described above.

● The presence of other funders already supporting CCM in relevant districts. The results table includes information on funders in each district so that PMI can factor this into its decisions.

In addition, a few additional limitations exist when comparing this CCM Malaria model to other models. These factors would not affect the prioritization of districts for CCM Malaria, but they could affect how CCM compares to other interventions. These include:

● Average cost of CCM - Our analysis of costs is limited due to lack of context-specific information on costs to PMI of community case management in Uganda.

● Effect on malaria mortality - Our analysis of the overall mortality effect of CCM is preliminary, so this factor should likely be revisited in more depth before relying too heavily on these results in prioritizing across different interventions.

● Effect on other causes of child mortality - This model considers the role of community case management in reducing under 5 child mortality due to malaria. It does not consider additional benefits of community case management for other leading causes of child death, such as pneumonia and diarrhea. Our current understanding is that the districts prioritized here do not yet have funding for integrated community case management.

● Conservatism - GiveWell adjusts cost-effectiveness estimates for quality of the evidence base (internal validity) and how that evidence may translate to new settings (external validity). For this and other

21 The data is available in the map of "ICCM Coverage in Uganda," Pg. 20, Global Fund "Malaria Funding request" document.

https://givewell.app.box.com/file/658648678840 https://givewell.app.box.com/file/658648678840 reasons, GiveWell’s cost-effectiveness models tend to be more conservative than some other published cost-effectiveness estimates.

Comparing approaches to prioritizing districts

Prioritizing based on cost-effectiveness is just one among a number of approaches to selecting geographic areas.

For example, in Uganda's Global Fund proposal, CCM districts were prioritized based on a set of criteria that considered five different factors, including poverty, access, burden and poor health sector performance.[5]

We ran simulations comparing the approach proposed in this report, to two other approaches that could be used to select districts: Targeting high-burden districts using DHIS2 data22; and targeting poor-performing districts based on a weighted index--the Malaria League Table.23 We compared these targeted approaches to a non-targeted approach, where districts are selected at random. In all simulations we assume a fixed budget that allows for the selection of 20 districts.

All approaches to targeting are more cost-effective than a non-targeted approach. However, the prioritization approach proposed in this report is roughly 6.8x more cost-effective than a non-targeted approach. The next-best approach is only 2.6x more cost-effective than no targeting. If we take these results at face value, that means that more than twice as many lives could be saved by targeting districts based on cost-effectiveness, rather than by targeting districts based on DHIS2-measured malaria burden.

These results should not be interpreted literally. These simulations assume the cost-effectiveness model is a correct representation of the world. In addition, it is not a surprise that this model outperforms other approaches--the approach proposed here both defines a model of cost-effectiveness and then optimizes against it. Nevertheless, the degree to which this model outperforms other approaches suggests that the model would have to be substantially wrong for these results to be overturned.

22 Specifically, we use the number of DHIS2-reported malaria cases at the village level for children under 5. Results are robust to other measures of burden, (i) including population-adjusted case rates, (ii) malaria mortality, as estimated by the Malaria Atlas Project; and (3) malaria cases reported at health facilities.

23 The Malaria League Table assigns a score for districts based on four indicators measured under the Uganda Malaria Reduction Strategic Plan. The four indicators are (i) Percent of Malaria Suspects tested for Malaria (ii) Percent of Negative Malaria cases treated using ACTs (iii) percent of Pregnant women receiving IPTp3 (iv) Percent of Pregnant women receiving LLINs at ANC1. Districts under-performing in each indicator received a lower score in the Malaria League Table.

Endnotes One More on the basics of the VHT program: "Uganda, an East African nation home to nearly 40 million people, is one of many countries facing a critical shortage of health workers and a heavy burden of morbidity and mortality

[1, 2]. Low- and middle-income countries (LMICs) have responded to these shortages by utilizing community health workers, a diverse group of health workers equipped to provide basic health interventions in local neighborhoods and communities [3]. Uganda founded the Village Health Team community health worker program in 2001 in an effort to bridge the gap created by the health worker shortage and improve equity in access to health services [4]. Within Uganda’s tiered national health system, the Village Health Team holds the position of Health Centre I, followed by Health Centres II-IV, which are local clinics, each with sequentially higher levels of capacity and larger catchment areas [5]. Similar to other types of community health workers (CHWs) throughout the Global South, Village Health Team members (VHTs) “mobilize communities for health programs and strengthen the delivery of health services at household level” [3, 4]. After basic training, each VHT member is assigned a cohort of households in his or her home community. VHT responsibilities include recording demographic and health data, educating on health and hygiene topics, mobilizing families to engage in health programs such as vaccination campaigns, monitoring for illness, making referrals, and providing post-discharge follow up [4]. Since the program’s inception in 2001, over 179,000 individual VHTs have been trained and, as of

2015, VHTs operate in all of Uganda’s 112 districts [4, 5]. Distinct among some CHW programs, VHTs are unpaid volunteers who typically maintain other daily occupations such as farming or shopkeeping [3, 5]." Mays et al.

For confirmation that VHTs can provide treatment, see: "Participatory training workshops in malaria case management were held for CHWs in each arm, based on a trainer’s manual and accompanying set of pictorial job aids (available from www.actconsortium.org/RDThomeman agement). Training workshops lasted 3 days for

CHWs in the control arm and 4 days for CHWs in the mRDT intervention arm. The latter received an additional

1-day training to cover the rationale for diagnostic testing in febrile patients, performing an mRDT and interpretation of the test result based on training materials from WHO [12]. CHWs in the intervention arm were trained to treat children with a mRDT-positive test and symptoms of uncomplicated malaria with artemether– lumefantrine; and to treat mRDT-positive children with danger signs with rectal artesunate as a pre-referral treatment, and refer these children to the nearest health unit." Pgs. 1158-1159, Ndyomugyenyi et al. 2016

Two Malaria Atlas Project (MAP) uses the following methodology to estimate local-level Plasmodium falciparum malaria incidence and mortality rates in sub-Saharan Africa:

● MAP extracts data on the prevalence rate of P falciparum parasites by country from the Demographic and Health Survey Program (DHS) and other surveys.24

● MAP uses parasite rate data and a set of covariates in a predictive model to estimate P falciparum prevalence rates for children aged 2 to 10 years by year and location (5 kilometer by 5 kilometer area)

24 "This approach relied on household surveys collected for clusters of households at fixed geographical locations. The P falciparum parasite rate points came from national surveys collected by organisations such as the Demographic and Health Survey Program, and through systematic literature reviews of P falciparum prevalence studies. The resulting dataset consisted of 43187 P falciparum parasite rate points collected from 2000 to 2017 and included data from 43 of 45 countries in sub-Saharan Africa (appendix p 4)." Weiss et al. 2019, Pg 324.

https://equityhealthj.biomedcentral.com/articles/10.1186/s12939-017-0619-6 https://equityhealthj.biomedcentral.com/articles/10.1186/s12939-017-0619-6 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5031222/pdf/TMI-21-1157.pdf https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736(19)31097-9.pdf across sub-Saharan Africa. Covariates include data on malaria interventions and data on precipitation and temperature.25 Parasite prevalence rates are aggregated from 5x5 km areas to national and subnational areas.26

● MAP then converts data on parasite prevalence rates to estimates of malaria incidence and mortality.27

Three Quantitative adjustments may not have seemed appropriate for reasons such as:

● It is sometimes unclear whether a particular factor should lead to a positive or negative adjustment. For example, working in the same district as another funder might provide the opportunity to leverage their funding. On the other hand, that district may be less neglected since another funder is supporting it. In this case it seems most appropriate to include the information so that PMI can consider this on a case-by-case basis.

● We want to avoid "double counting." For example, poverty rates are probably associated with lack of access to healthcare and malaria burden, but those factors are already quantitatively adjusted for in our model. Nevertheless, a funder may want to independently take poverty rates into account to some extent.

25 "The P falciparum parasite rate points were standardised to the 2–10 years age range (P falciparum parasite rate) and to account for differing diagnostic methods. The predictor data consisted of geospatial environmental and socioeconomic data characterising vector and human habitats. The predictor dataset also included geospatial surfaces depicting temporally changing coverage of the main malaria control interventions such as insecticide treated bednets, indoor residual spraying, and effective treatment with an antimalarial drug. Each of the interventions was modelled independently and included to improve estimates and spatiotemporal predictions of P falciparum parasite rate. These data were used in a Bayesian space-time geostatistical model to predict P falciparum parasite rate 2–10 for every 5×5 km pixel across all of sub-Saharan Africa annually from 2000–17 (appendix p 8)." Weiss et al. 2019, Pg 324.

"Some covariates represent variables that vary over time, such as temperature. We hold these for time-steps as frequent as 8 days (although generally we produce our models at monthly or annual time-steps). We refer to these as dynamic covariates. Others are “fixed” in time (at least for the purposes of our work), such as elevation; we refer to these as static or synoptic covariates. Many covariate datasets are acquired from third-party sources such as NASA and the WorldPop project; others are developed in-house by MAP’s scientists. Datasets include land surface temperature, vegetation health indices, precipitation, land cover types, population maps, and night-time lights. Malaria Atlas Project Malaria Covariates page.

26 "The model allowed predictions to be made at any location by drawing strength from nearby P falciparum parasite rate observations, observations from earlier years, and by the relationships between those observations and the suite of covariates. We generated stochastic realisations from the model’s posterior distribution, reflecting uncertainty due to availability and spatial heterogeneity within the P falciparum parasite rate points. The realisations also provided the basis for generating estimates of P falciparum parasite rate aggregated across national and subnational administrative units."

Weiss et al. 2019, Pg 324 27 "P falciparum incidence was calculated by converting P falciparum parasite rate into P falciparum incidence in three age-groups (0–4 years, 5–14 years, and 15 years and older) using an established relationship. This grouping reflected the biological relationship between the cross-sectional prevalence of infection in a given community and the resulting incidence of uncomplicated clinical malaria. This relationship was captured using an ensemble of mechanistic transmission models calibrated against observations at 30 sites from 24 studies that reported age-specific incidence of clinical malaria fevers and local P falciparum parasite rate.22 Statistical uncertainties from the calibration procedure were propagated through to predictions from this ensemble, which also allowed adjustments for the local effective treatment rate (proportion of individuals with a fever who seek treatment) and the seasonality of transmission.P falciparum mortality was calculated using a combination of the pixel-level and case fatality rate (CFR) model for sub-Saharan Africa. The CFR modelling approach built on earlier work and used updated cause of death studies (verbal autopsy and vital registration reports) extracted from the GBD database." Weiss et al. 2019, Pg 324.

https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736(19)31097-9.pdf https://malariaatlas.org/data-project/covariates/ https://malariaatlas.org/data-project/covariates/

Four LiST is a tool to estimate the impact of life-saving interventions in developing countries. LiST does not use simulations or dynamic disease transmission models—instead, it makes linear adjustments for mortality estimates based on intervention coverage.28

How the Lives Saved Tool combines the impact of multiple interventions:29

"Within LiST, the efficacy of an intervention is defined in terms of the reduction of a cause of death or risk factor.

The calculation of impact is simple when considering only a single intervention, because the change in coverage times the efficacy of the intervention is applied to the level of cause-specific mortality. For example, there may be 10,000 diarrhea deaths in children aged 1-59 months, and the proposed intervention is the introduction of a new vaccine that would be 50%effective in reducing diarrhea mortality. If coverage reaches 50%, diarrhea mortality among children would be reduced to 7,500 (=10,000 – (10,000 *0.5 *0.5)). If a second or a third intervention is added, the same approach is followed, although the impact of the additional diarrhea intervention(s) would be applied to the residual diarrhea deaths. Following the previous example, if the second new diarrhea intervention is also 50% effective and coverage reaches 50%, diarrhea mortality would fall to 5,625

(=7,500 – (7,500 *0.5 *0.5). By using cause-specific efficacy and applying each intervention to the residual deaths remaining after the previous intervention, LiST ensures that double counting is avoided and the potential impact of multiple interventions is not erroneously inflated."

Based on the description of LiST's methodology above, it is our understanding that accounting for baseline malaria mortality rates in non-dynamic models like LiST is functionally equivalent to accounting for the impacts of current malaria interventions. Based on these assumptions, we expect that including both malaria mortality rates and LLIN coverage estimates would be "double-counting" the same result.

Five "Selection of districts for iCCM scale up: iCCM is implemented to ensure benefit for the underserved children in the country. The selection of the districts is based on a number of criteria, i.e. (i) a high poverty index and high absolute number of under-five children dying; (ii) poor access to health facilities as measured by the number of public and private facilities relative to the district population; (iii) a high burden of malaria diarrhoea or pneumonia, (iv) poor performance in the MOH district annual health sector league table; and (v) other donor programmes that could be leveraged. These parameters were weighted before the final selection."

28 "LiST has been characterized as a linear, mathematical model that is deterministic [12]. It describes fixed relationships between inputs and outputs, and the tool will produce the same outputs each time the model is run with identical inputs.

For LiST, the primary inputs are coverage of interventions and the outputs include changes in population-level risk factors

(e.g., wasting or stunting rates, birth outcomes such as prematurity or size at birth) or cause-specific mortality (e.g., neonatal or child (1-59 months), maternal mortality and stillbirths).The relationship between changes in inputs

(intervention coverage) and one or more outputs is specified in terms of the effectiveness of the intervention for reducing the probability of that outcome. Outcomes of interest include either cause-specific mortality or a risk factor for mortal-ity

(e.g., intrauterine growth restriction or stunting). The overarching assumption in LiST is that mortality rates and the cause of death structure will not change dynamically, and that any differences will be solely in response to changes in intervention coverage." Walker et al. 2013, Pg 2 29 Walker et al 2013, Pg 2 https://bmcpublichealth.biomedcentral.com/track/pdf/10.1186/1471-2458-13-S3-S1 https://bmcpublichealth.biomedcentral.com/track/pdf/10.1186/1471-2458-13-S3-S1

Pg. 20, Government of Uganda, “Global Fund Malaria Funding request" draft document.