RFA-CE-24-012 Questions and Answers_final.pdf

RFA-CE-24-012; The CDC’s National Center for Injury Prevention and Control | Rigorous Evaluation of Policy-Level Interventions to Prevent Overdose (R01) | Questions and Answers

What level of statistical expertise is required to be considered responsive? Are biostatisticians more favorable team partners than an economist?

• Answer: CDC encourages professionals from a wide spectrum of disciplines including epidemiology, behavioral and social sciences, medicine, biostatistics, economics, urban planners and others to conduct research that informs violence prevention efforts. Per the NOFO Section I. Funding Opportunity Description, 2. Approach, Research Design on page 14, this funding opportunity is intended to support evaluations that are designed to detect whether any measured changes in primary overdose-related outcomes can be attributed to the selected policy. To this end, applicants are expected to employ the most rigorous research design feasible to examine the impact of federal, state, local, tribal, or organizational policies on primary overdose-related outcomes and inequities in rates of overdose. For this announcement, rigorous evaluation designs include those that utilize experimental (i.e., randomized controlled trials) or quasi-experimental designs (e.g., comparative interrupted time series design, difference-in-differences, instrumental variable methods, regression discontinuity, regression point displacement, stepped wedge, designs using propensity-score matching, designs involving matched comparison groups). Because a policy must have been previously or is currently implemented and may have been implemented outside the control of the investigators, applicants must propose to utilize an appropriately matched comparison group or other design that would maximize the ability to make causal inferences from the findings. Applicants are expected to justify that the proposed design and data analysis plans (including attention to statistical power and baseline measures) are appropriate for evaluating the impacts of the strategy.

• Per the Data Collection and Acquisition section on page 15 of the NOFO, it is incumbent on the applicant to explain all appropriate data sources to be used to support their proposed research.

Applicants are not required to conduct primary data collection and using existing data sources is preferred. However, applicants proposing to collect data, for example, to triangulate findings from secondary data sources, or any new collection should describe how the collection will be conducted, and all proposed analyses completed within the project period with consideration for community collaboration (e.g., community-based participatory research, inclusion of lived experiences). It is also incumbent on applicants to describe the proposed processes, partnerships, and resources required to accomplish data collection and management.

Applications that propose law enforcement without a public health component will be considered nonresponsive; would CDC be able to provide examples of this?

• Answer: The NOFO provides additional information on policy selection Section 1. Funding Opportunity Description, 2. Approach, Policy Selection on page 11. As a public health agency, CDC focuses on public health responses to overdose prevention. While overdose prevention strategies may collaborate with law enforcement agencies, applications that propose to focus on law enforcement polices without a public health component or focus, as evidenced in the Research Strategy section of the application’s research plan, will be considered nonresponsive and will not be forwarded for peer review. An example of a policy not meeting the criteria for this NOFO includes mandatory minimum penalties for drug offenses.

Can CDC define the expectations around proposals evaluating multiple policies?

• Answer: It is acceptable to evaluate multiple policies. The Research Design section on page 15 states that while it can be difficult to distinguish the impact of a policy when more than one is implemented simultaneously (Haegerich et al., 2019), applicants are also encouraged to consider designs that allow comparing changes in outcomes across polices if more than one is selected, in particular those with different aims, where data will allow. If an applicant proposes to evaluate more than one policy, each policy must be evaluated for its impact on at least two primary overdose-related outcomes and inequities across populations of interest (the latter if the data allow it). The Approach section of the Scored Review Criteria on Page 38 of the NOFO outlines that if two or more policies are being evaluated however, each policy should be evaluated of its impact on reducing rates of two or more primary overdose related outcomes independently of the other. Please refer to Policy Selection on page 11 for more information.

What is the CDC’s definition of policy?

• Answer: The NOFO provides additional information on policy in Section 1. Funding Opportunity Description, 2. Approach, Policy Selection on page 11. This NOFO intends to evaluate policies that have the potential to affect social and structural conditions to reduce inequities in rates of overdose across population groups. This means the policy applies broadly to the level at which it is enacted (i.e., local, state, tribal, federal, or organizational policy [e.g., school or health systems]) and included in the evaluation are primary overdose-related outcomes at the same level. The policy selected for evaluation must have been implemented before the proposed period of evaluation. Per page 11 of the NOFO, the proposed research should focus on creating new knowledge about the impact of policies aimed at reducing or regulating substance use or drug overdose. Per page 12 of the NOFO, this NOFO will not support applications that propose to evaluate strategies (e.g., media campaigns or educational interventions) that only address individual-, peer-, or other proximal relationship and family-level factors (e.g., parenting skills, personal attitudes, knowledge about substance use, and overdose prevention). In other words, evaluating outcomes among participants of a program mandated by a policy would not be consistent with the intent of the NOFO. Examples of polices that would be considered responsive are listed at the end of page 12 for more information. Please also see CDC’s definition of “policy” for more information.

https://www.cdc.gov/policy/paeo/process/definition.html

What level of detail for policy implementation should be provided?

• Answer: The purpose of this NOFO is to rigorously evaluate effectiveness of policies at reducing substance use or drug overdose. This is not looking at the implementation science of a policy.

Process or implementation measures can include but are not limited to barriers and facilitators to implementation of the policy, extent to which the policy was implemented as enacted and acceptability by the population impacted by the policy. The narrative describing the logic model or theory of change should describe the level of policy implementation and intended reach (e.g., population(s), geographic region) and level of evaluation that is appropriate.

• Please review Objectives/Outcomes on page 16 for more information. The primary goal of this research is to evaluate the impact of a selected local, state, tribal, federal, or organizational policy (e.g., school or health systems) aimed at reducing or regulating substance use or drug overdose, on reducing rates of at least two primary overdose-related outcomes.

How should implementation be incorporated into the logic model?

• Answer: Process and implementation measures are included in the objectives because it is equally important for applicants to quantify impact of policies on health outcomes as well as understand how implementation affects achieving intended outcomes of policies among groups experiencing a disproportionate burden of overdose and/or groups that are at greater risk of experiencing adverse outcomes related to substance use due to social determinants of health.

Implementation measures, such as barriers and facilitators to implementation of the policy, the extent to which the policy was implemented as enacted, and acceptability by the population impacted by the policy can aid the understanding of the differential impacts of policies.

• Applicants are strongly encouraged to further examine differences in outcomes by one or more social determinant of health measures and provide information on the importance of the selected measure in the context of overdose, particularly for policies that have been studied and would not be considered as creating new knowledge without measurement and analysis of social and structural determinants. Secondary outcomes, such as mediators and moderators, can identify associations between primary outcomes and policy outcomes and are encouraged in the identification of data sources in the proposed evaluation when applicable. Please refer to Logic Model or Theory of Change on pages 13 and 14 for more information.

Regarding our application, we are interested in better understanding CDC’s preference towards language used around overdose to be considered responsive?

• Answer: We are unable to provide specific guidance or information on the appropriateness of your aims to the NOFO. We encourage you to review the Executive Summary beginning on page 3, the Approach beginning on page 11, and Responsiveness listed on page 23 for more information responsiveness criteria.

Are there are any preferences in proposed policy usage? Would research evaluating more recent policy considered more responsive?

• Answer: The NOFO provides additional information on policy selection Section 1. Funding Opportunity Description, 2. Approach, Policy Selection on page 11. Applicants are encouraged to propose rigorous evaluations of policies that do not already have demonstrated impacts on primary overdose-related outcomes in the scientific literature. These include previously or currently implemented policy strategies with promising or undetermined evidence of impact on primary overdose-related outcomes. Promising policies include those with demonstrated impacts on risk and protective factors for primary overdose-related outcomes, but not on these specific outcomes. Policy strategies that have not been evaluated with an experimental or quasi-experimental design are considered to have undetermined effectiveness. The policy selected for evaluation must have been implemented before the proposed period of evaluation and may or may not be ongoing.