Appendix A- Methods Guide 8.31.22.docx

Methods Guide: Instructions for CMS CAG Environmental Scans

Contents

The Methods Guide is a blend of standard operating procedures (SOPs) and templates for mature, emerging, and emerging with evidence synthesis environmental scans (ES). Note that for emerging scans, this guide should be paired with the NTAP or Evidence Preview templates since content is populated directly into those documents. For these emerging scans that warrant direct population of a template, not all sections of this guide will be relevant. If a specialist physician has been included on the review team, the physician should assist with development of the appropriate search strategy, review all language in the report for accuracy, and incorporate clinical context particularly related to the clinical importance of any findings. The clinical specialist should ensure that the report contents are consistent with contemporary clinical standards and appropriately reflect important clinical considerations.

STEP 1: PROJECT DEFINITION

Within the initial step, the CAG defines the topic and key questions (KQs) for the contractor and proposes eligibility criteria (PICOT) for identification of the “core” body of evidence in the form of Table 1 below. The core body of evidence is generally comprised of pivotal randomized, controlled studies that form the basis of FDA market authorization and/or expansion of indication. The contractor performs an initial search based on the proposed criteria and can provide feedback to CAG if search results are either too limited or too expansive (aim for ≤50 citations that potentially meet PICOT criteria, according to Title/Abstract review, for a scan of a mature technology and < 20 for a scan of an emerging technology). If so, then the contractor and CAG can discuss modifications to the PICOT criteria. Restrictions to narrow the search to a reasonable volume may be made based on publication date range, sample size, duration of follow-up, setting, study design, or other factors. CAG approves the revised PICOT criteria that is appropriately scoped. Similarly, CAG and the contractor can refine the KQs if needed. The contractor will have the opportunity to discuss modifications to the criteria if needed based on preliminary searching. At the minimum, it is essential for CAG to provide the contractor with the topic, KQs, and PICOT criteria.

Note that if a systematic review (SR) generated for an existing NCD is available, then CAG can request that the contractor uses the same eligibility criteria from that review. The contractor should abstract the PICOT criteria and key questions from the existing NCD but should verify with CAG that they remain appropriate before initiating the SR.

Note that for emerging scans, CAG will generally provide the contractor with application materials, a draft or finalized Summary of Safety and Effectiveness Data (SSED) document, and/or a manufacturer-generated bibliography. The contractor is expected to supplement that data with a PubMed search to ensure all applicable published, peer-reviewed studies are included in the review.

Background and Objectives for the Systematic Review

Project Title: The title should succinctly indicate the interventions or exposures and the associated health or social problem addressed in the review. The title is typically listed in the initiation email from CAG.

Background:

The background section provides important context to the SR and is generally approximately 1 page in length and at a level of detail consistent with an Up-to-Date review; 3 or more paragraphs may be required, but a general format follows.

Paragraph 1: Brief overview of condition, epidemiology (incidence, prevalence), impact on patients Paragraph 2: Therapeutic options, unmet need Paragraph 3: Description of device, intended application, what value it could add to therapeutic or medical testing options, prior NCA decision (for mature ES, if applicable)

Objective: “The primary objective of this report is to […]”

Key Questions Define the KQs that will be evaluated as part of the SR. These questions should be specific high-impact questions that are relevant to decision makers. When appropriate, questions should clearly state the populations (and subgroups), interventions, and main outcomes of interest. KQs that must be addressed in the NCA include:

(1) Is the evidence sufficient to conclude that [topic] is reasonable and necessary for the treatment of target Medicare beneficiaries with [condition]?

(2) Is there evidence that specific characteristics or comorbidities make patients more or less likely to benefit from [topic]?

(3a) Is it likely that specific treatment conditions are necessary to achieve [topic] outcomes similar to those demonstrated in the clinical studies reviewed in this analysis?

(3b) Given the settings and treatment conditions under which the reviewed clinical studies were conducted, might healthcare disparities in access to [topic] or outcomes from [topic] be expected?

Note that these key questions are those that CAG staff will consider when reviewing the evidence; ICA is not expected to judge whether an item or service meets the “reasonable and necessary” standard as defined in statute.

STEP 2: EVIDENCE TABLE REFINEMENT

The contractor will adapt the template in Appendix B to describe study characteristics and outcomes for each individual study included in their systematic review. The template is structured to provide information about the individual study’s design, quality, interventions assessed, target population, as well as findings of the individual studies for the specific outcomes. This table allows the reader to assess the generalizability of the study to the Medicare population, as well as determine which studies are applicable to specific outcomes of interest. Additional tables driven by the included studies or heterogeneity of the outcomes measures can be also developed.

In the template, we include the contents of each column with the suggested options or clarifying information indicated within [ ], as well as a sample row from the Pharmacogenetic Testing for Warfarin Response environmental scan. Note that prior to populating this table, the systematic review team will include information about study design requirements or populations and outcomes of interest that had been previously defined in the PICOT table. Within the template, the team will also discuss the quality of the individual studies as described under “Quality Assessment of Individual Studies”.

During this step, CAG will have the opportunity to suggest modifications to the evidence table template if necessary. The modifications will highlight domain/intervention specific outcomes or subgroups of interest that should be abstracted during the ES. The CAG team is encouraged to use a pivotal trial or discussions with stakeholders to guide the suggested template modifications. Additional work by the CAG team during this step will minimize the need to revisit evidence downstream to support the needs of decision makers.

STEP 3: ENVIRONMENTAL SCAN

Within this step, the Contractor performs an ES using at least one database (PubMed) with an additional database listed (EMBASE or other context-specific database) identified during the evidence analysis step if appropriate. The Contractor then screens the available published literature and populates the PICOT table with the approved inclusion/exclusion criteria (Table 1). The contractor will also populate the evidence table(s) that include individual patient and study characteristics, interventions, and study outcomes (Appendix B). Search tables and a PRISMA diagram are also produced (Appendix A), along with an Excluded Studies Table (Appendix C). Study quality is assessed via RoB tools best suited to study design. In the case of an all-observational data set, no formal RoB assessment will be performed and the evidence base will be deemed Unclear to High RoB. Strength of evidence (SoE) is assessed in emerging ES with evidence synthesis using the GRADE methodology.

Regarding SRs or meta-analyses (MTAs), the Contractor should use high-quality SRs/MTAs to cross-validate the primary studies when possible. CAG considers the totality of the evidence when making national coverage determinations. Other types of studies that do not qualify per the pre-defined PICOT criteria, such as registry, single-arm studies in the case of a PICOT requiring only comparative studies, or real-world evidence studies, should be flagged for CAG’s attention. To facilitate efficient completion of NCD reviews, an EndNote file containing these flagged, supplementary studies will be provided to CAG along with the final report.

For emerging environmental scans, the CAG team will review the SR and perform a gap analysis relative to published reasonable and necessary criteria. CAG will them complete either the NTAP or Evidence Preview template by adding a brief summary of the evidence along with any identified specific evidence deficiencies. These reports are used to assess eligibility for various programs and/or to provide feedback to manufacturers on further evidence development requirements. For mature technologies, CAG will incorporate elements of the SR into the NCD template to accelerate development of evidence-based policy. Mature SRs are most useful where specific sections, paragraphs, and tables are highly standardized and can be integrated into the NCD with minimal editing.

Inclusion and Exclusion Criteria Identify the population, interventions, comparators, outcomes, timing, and settings (PICOTS) for each KQ as they pertain to the decisional dilemmas – preferred format includes both inclusion and exclusion criteria in a table (Table 1). Discuss any additional information or changes to the PICOTS criteria made by the contractor team during execution.

Table 1. Inclusion and exclusion criteria

· Important subgroups of interest should be specifically listed

· Note if there are differences in populations of interest by key question Any populations who are excluded from the review should be indicated here

· List medications by generic name, with class of drug and other relevant details. Include dosage, frequency, and methods of administration if applicable

· Describe nonpharmacologic interventions.

· For devices, list type with relevant key features or characteristics.

· For bundled interventions, define the components of the intervention.

· Describe co-interventions, if any.

· Note if there are differences in interventions of interest by key question Provide details on interventions not of interest

· Note if there are differences in comparators of interest by key question Provide details on comparators not of interest

· Specify primary and secondary outcomes of interest—including harms

· List potential serious adverse effects (outcomes), including intervention-related adverse or side effects

· Clearly define intermediate outcomes, if relevant.

· If instruments or scales have been validated for outcome assessment, identify them.

· Note if there are differences in outcomes of interest by key question Study does not include any outcomes of interest

· Note if there are differences in timing by key question Studies which do not align with identified duration of follow-up

· Clearly define target geographic locations

· Note if there are differences in setting by key question Studies which were conducted in settings outside those identified as of interest

· Study design (RCTs, prospective observational, retrospective observational)

· Required sample size

· Inclusion of existing systematic reviews or meta- analyses Studies which did not meet the required study design or sample size

· Peer-reviewed publication

· Indicate if there are additional restrictions in terms of publication date (i.e., published on or after a specific date) Non-English language publications

Abstracts, conference proceedings, gray literature, publications outside publication type, publications outside date range

Identifying the Literature and Study Selection The contractor will create the following paragraph of the final literature search, as well as describe screening methods (single vs. dual, QC, cross-validation, etc.). The contractor will also include a literature flow figure/PRISMA diagram that describes the literature identified through the searches, the exclusions (with reasons at the full-text level), and then the included studies for the initial triage decision (Appendix A). A separate table for excluded studies will also be generated to document the reasons for exclusion of individual studies at the full-text level (Appendix C). The contractor will create paragraphs similar to those below that summarize this process and narratively describe the PRISMA diagram:

“We searched databases [list databases] on [date] using a combination of key words along with their synonyms and Boolean operators and combined search terms as detailed in Appendix A for studies published between [date] and [date]. This search identified [x] records. We applied the exclusion criteria as detailed in Table 1. After applying exclusion criteria, we included [y] full text articles reporting on [z] studies.

Collectively, we identified XXX articles from our database searches and SR cross-validation. Deduplication excluded XXX records, and XXX were excluded due to reasons x, y, and z at the title/abstract level. Full-text identification successfully retrieved XXX of XXX records, after which full-text screening excluded XXX records due to reasons x, y, and z. The final core evidence base consists of XXX articles.”

Data Abstraction and Management Describe data abstraction methodology, study-level and patient-level characteristics captured, and outcomes of interest. Reference evidence tables in Appendix B.

Summary of Included Studies Below is an example paragraph summarizing the included studies for a systematic review. The contractor will create a similar paragraph describing the included studies in the review:

“We identified [y] articles representing [z] studies relevant to the KQs. Table 2 describes the size of the evidence base for each KQ. A total of [xxx] RCTs and [xxx] observational studies were included in our analyses. The included studies explored interventions in studies of diverse quality, funding, and geographical location. Additional study characteristics can be reviewed in Appendix B. Regarding funding, [xxx] studies were sponsored solely by industry, [xxx] by government, [xxx] received funding from non-government, non- industry sources, [xxx] received funding from multiple sources including government, industry, non-government and non-industry, and [xxx] had either no sponsorship or this information was unclear. Among the [z] studies, [xxx] were performed in the UK or Europe, [xxx] in the United States, [xxx] in Canada, and [xxx] were conducted on multiple continents. A total of [xxx] studies were unclear or did not report a geographical location. [xxx] studies were considered good quality, [xxx] were considered fair quality, and [xxx] were of poor quality.”

Table 2. Evidence Base for Key Questions

Example: 8 RCTs, 2 subgroup analyses

*Some studies had data relevant to more than one KQs Abbreviations: KQ: key question; RCT: randomized controlled trial; [insert more abbreviations as needed]

Key Question 1 Summarize findings for each KQ.

Key Question 2 Summarize findings for each KQ.

Key Question 3 Summarize findings for each KQ.

Quality Assessment of Individual Studies The methodological quality, or risk of bias (RoB), for randomized and observational individual study designs will be assessed using validated RoB tools appropriate to the study’s design (e.g., CLARITY (clinical advances through research and information translation) Group at McMaster University and USPSTF). The following domains, as appropriate for each study design, include: (1) random sequence generation, (2) allocation concealment, (3) blinding of participants and personnel, (4) blinding of outcome assessors, (5) incomplete outcome data, (6) selective reporting, (7) attrition, and (8) other biases. Briefly, each study is judged as having the potential for “high RoB”, “moderate RoB”, or “low RoB”. For more information on the CLARITY tool visit: CLARITY Risk of Bias Commentary.1 For more information on the USPSTF tool visit: USPSTF Assessing Internal Validity of Individual Studies.2

RoB is assessed by one systematic reviewer and reviewed by a senior team member. Table 3 describes the overall study quality assessment ratings. All Investigator attempts to mitigate potential sources of biases will be documented.

Table 3. Definitions of overall quality ratings

Good (Low RoB) These studies have the least bias, and the results are considered valid. These studies adhere to the commonly held concepts of high quality, including the following: blinding of outcome assessors, low attrition, no indication of selective outcome reporting, study was not stopped early, and no large differences in baseline characteristics between study arms, no suggestion of fraudulent scientific methods. For RCTs, the study authors used adequate randomization methods and allocation concealment.

problems.

large amounts of missing information; or discrepancies in reporting.

The grading is study-specific such that different quality grades are assigned for individual studies. Studies of different designs are graded within the context of their respective designs. Thus, RCTs are graded good, fair, or poor, and observational studies are separately graded good, fair, or poor. If an evidence base is made up of all observational studies, no formal RoB assessment will be performed, as observational studies are generally considered to have High RoB. A statement will be added that describes any methodological steps that were taken to reduce the RoB.

Example questions from these tools are included below:

CLARITY in Randomized Controlled Trials (Clinical Advances through Research and Information Translation Group at McMaster University) Indicate Definitely Yes, Probably Yes, Definitely No, or Probably No for the following:

· Was the allocation sequence adequately generated?

· Was allocation adequately concealed?

· Was knowledge of the allocated interventions adequately prevented?

· Were patients blinded?

· Were healthcare providers blinded?

· Were data collectors blinded?

· Were outcome assessors blinded?

· Were data analysts blinded?

· Was loss to follow-up (missing outcome data) infrequent?

· Are reports of the study free of suggestion of selective outcome reporting?

· Was the study apparently free of other problems (e.g., stopped early, extreme baseline imbalance) that could put it at a risk of bias?

· Overall Bias

· Risk of Bias Judgment:

· Low for all key domains

· Unclear for ≥1 key domain

· High for ≥1 key domain

· Comments

USPSTF Criteria for Assessing Internal Validity of Individual Studies Select Good, Fair, or Poor based on the following criteria for RCTs and cohort studies:

· Good – meets all following criteria:

· Comparable groups are assembled initially and maintained throughout the study (follow-up greater than or equal to 80%

· Reliable and valid measurement instruments are used and applied equally to all groups

· Interventions are spelled out clearly

· All important outcomes are considered

· Appropriate attention to confounders in analysis

· Intention-to-treat analysis is used for RCTs

· Fair – any or all the following problems occur, without the fatal flaws noted in the "poor" category below:

· Generally comparable groups are assembled initially, but some question remains whether some (although not major) differences occurred with follow-up

· Measurement instruments are acceptable (although not the best) and generally applied equally

· Some but not all important outcomes are considered

· Some but not all potential confounders are accounted for

· Intention-to-treat analysis is used for RCTs.

· Poor – any of the following fatal flaws exists:

· Groups assembled initially are not close to being comparable or maintained throughout the study

· Unreliable or invalid measurement instruments are used or not applied equally among groups (including not masking outcome assessment)

· Key confounders are given little or no attention

· Intention-to-treat analysis is lacking for RCTs.

· Comments

Applicability The contractor will also assess the applicability of the included studies to the Medicare population of interest. Specifically, following the guidance from AHRQ’s Methods Guide (AHRQ 2014)3 we use the PICOTS format as a way to organize information relevant to applicability. Factors such as how closely demographic features of the enrolled study populations compare to the target Medicare population, with special emphasis on age, comorbidities, and country of origin, how well the delivery of study interventions compare with how care is typically delivered in the community setting, and if only surrogate outcomes are reported by study authors will be considered. Issues of applicability will be summarized qualitatively. See below for more detail on assessing applicability.

Applicability Guide Use the PICOTs format to identify specific issues, if any, that may limit the applicability of the study to this review.

· Population (P)

· Narrow eligibility criteria and exclusion of those with comorbidities

· Large differences between demographics of study population and community patients

· Narrow or unrepresentative severity, stage of illness, or comorbidities

· Run-in period with high-exclusion rate for nonadherence or side effects

· Event rates much higher or lower than observed in population-based studies

· Intervention (I)

· Doses or schedules not reflected in current practice

· Monitoring practices or visit frequency not used in typical practice

· Older versions of an intervention no longer in common use

· Cointerventions that are likely to modify effectiveness of therapy

· Highly selected intervention team or level of training/proficiency not widely available

· Comparator (C)

· Inadequate comparison therapy

· Use of substandard alternative therapy

· Outcomes (O)

· Composite outcomes that mix outcomes of different significance

· Short-term or surrogate outcomes

· Setting (S)

· Standards of care differ markedly from setting of interest

· Specialty population or level of care differs from that seen in community

· Comment

Discussion 08/31/22

Summarize findings and other considerations to contextualize findings.

STEP 4: EVIDENCE-BASED GUIDELINES

For mature scans, the contractor will identify and list any relevant evidence-based guidelines that apply to the item or service under review. These are often published by specialty societies and expert working groups. Additionally, existing appropriate use criteria should be identified and documented in this section.

Evidence-based Guidelines

Professional Society Recommendations / Consensus Statements / Other Expert Opinion The contractor will identify and list any relevant professional society recommendations, consensus statements, or other expert opinion that apply to the item or service under review.

Appropriate Use Criteria The contractor will identify whether any appropriate use criteria exist for the item or service under review.

Step 5: Commercial Payer Survey A commercial payer survey of the five largest U.S. commercial payers should also be completed, which captures existing commercial payer policies that apply to the item or service under review. The payer survey does not require an extensive literature review; rather, a targeted search via Google or other general search engine is sufficient as the goal is to retrieve information that is publicly available. CMS recently explored whether commercial payer policies should be explicitly considered when making national and local coverage determinations. To be considered, commercial payer policies must be applicable to a broad population of covered lives and not a narrowly targeted one. For example, a plan directed specifically to senior corporate executives would be excluded. The commercial payer survey would also exclude Medicaid managed care, Medicare Advantage, and other government administered healthcare coverage programs as these enrollees are not considered to be in the commercial market.

As of 08/20, the top five commercial insurers include: (1) United Healthcare; (2) Cigna; (3) Anthem; (4) Humana; (5) Aetna.

United Healthcare

patients]

requirements]

STEP 6: EVIDENCE SYNTHESIS & ANALYSIS

For environmental scans that move forward to the evidence synthesis and analysis step, the following evidence summary table will be created summarizing both qualitative and quantitative findings from the evidence synthesis. Summary text describing the key findings is also outlined. The GRADE approach will be used to assess SoE.

Summary Text When summarizing the findings of the evidence analysis, CAG will adhere to the GRADE guidance4 on plain language (https://www.sciencedirect.com/science/article/pii/S0895435619304160) presented in the table below.

08/31/22

Evidence Summary Tables Evidence summary tables for each key question will be developed (Appendix D), along with a table detailing the overall assessment of quality of the evidence base for each major outcome (Appendix E). These summary tables will indicate the category of outcomes assessed, the specific outcome of interest, a summary of the included studies which assessed that outcome, the overall conclusion, and then the SoE and any rationale for that SoE rating.

Strength of Evidence Rating The strength of evidence (SoE) is assessed for each outcome. The SoE will be assessed using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach, as it is the most widely recognized system. In brief, the approach requires assessment of four domains: study limitations (previously named risk of bias), consistency, directness, and precision. Additional domains used when appropriate are dose-response association and magnitude of effect. When the body of evidence for a particular outcome included both RCTs and observational studies, grading will be done for each study type separately using design-specific criteria. In considering the overall strength of the entire body of evidence, consider the extent to which studies with higher risk of bias are consistent with studies that have lower risk of bias, particularly regarding direction and magnitude of effect. Because of the risk of unmeasured confounding, observational studies generally do not contribute to estimates of the magnitude of effect, and judgments about the precision of the effect, when RCT data are available. If there are other issues (such as differences in when and where RCTs were performed compared to observational studies, and how these differences might affect applicability), this would generally lead to increased uncertainty about the magnitude and precision of any treatment effect. These domains are considered qualitatively, and a summary rating of high, moderate, or low strength of evidence is assigned for each outcome by a single reviewer, with review by a senior reviewer.

· High—Team is very confident that the estimate of effect lies close to the true effect for this outcome. The body of evidence has few or no deficiencies. The team believes that the findings are stable, i.e., another study would not change the conclusions.

· Moderate—Team is moderately confident that the estimate of effect lies close to the true effect for this outcome. The body of evidence has some deficiencies. The team believes that the findings are likely to be stable, but some doubt remains.

· Low—Team has limited confidence that the estimate of effect lies close to the true effect for this outcome. The body of evidence has major or numerous deficiencies (or both). The team believes that additional evidence is needed before concluding either that the findings are stable or that the estimate of effect is close to the true effect.

· Insufficient to permit any conclusion—The evidence was too sparse and/or inconsistent to even suggest an estimate of effect.

STEP 7: UPDATING SCANS

Important studies are occasionally missed in an ES. For example, if a study was published shortly after a literature search was initiated, it would not be captured in the ES. Alternatively, if a gap of six months or more exists between the completion of an ES by the Contractor and the opening of a National Coverage Analysis, then it will be necessary to refresh the scan to capture critical, relevant studies published during that gap. The additional literature should be directly incorporated into the existing report, with the appropriate sections updated as needed (i.e., evidence tables and templated paragraphs).

Appendix A. Literature Search Strategy and PRISMA Diagram Table A1. Literature Search Strategies Literature searches were conducted on [date], using [one or two] electronic databases [list databases].

Filters:

#3

#2

#1

Filters:

#3

#2

#1

Figure A1. Literature Flow Figure “After removal of duplicates, the primary literature search identified a total of XXX records. A total of XXX papers were excluded at the title and abstract level, leaving XXX references reviewed at the full-text level. After XXX papers were excluded for reasons specified in the diagram below, XX papers were included in this review.”

Full-text articles assessed for eligibility N = Records excluded at title/abstract screening N = Full-text articles excluded (N = ) Reason #1 (n= ) Reason #2 (n= ) Reason #3 (n= ) Reason #4 (n= ) Reason #5 (n= ) Reason #6 (n= )

Included Eligibility Studies included in qualitative synthesis N = Screening Records identified through database searching N = Identification Additional records identified through other sources N = Records screened N = Records sought for retrieval N = Duplicate records removed N = Records not retrieved N =

Appendix B. Evidence Table Table B1. Study Characteristics and Outcomes

Reference: Author’s Last Name, Publication Year, Study Acronym (if applicable)

Country: [US/Canada/UK- Europe/S America/C America/ Asia/ Africa/Australia-NZ/ Unclear/Not Reported]

Study Design: [RCT/Prospective Observational/ Retrospective/Observational], [Single center/ Multicenter/ Unclear/Not Reported]

Purpose: [objective of study]

Study Quality1: [good, fair, or poor] [insert brief explanation for rating]

Funding Source: [Non-Government and Non Industry/Government/ Industry/Unclear/NR]

Notes: None Intervention: [Intervention/tool name]

Comparator: [Intervention/tool name]

Inclusion criteria: [Main study inclusion criteria or patient population]

Exclusion criteria: [Main study exclusion criteria]

Length of follow-up, mean: [Total follow up period. If not available, put “Unclear/NR”]

Patients (N): [Total and/or per Arm if reported as such]

Age, years, median years (range): [Put as reported, whether by arm or total. Include measure of uncertainty such as standard deviation (preferred) or range.]

Age ≥ 65, n (%): [If available note the proportion of patients included that are 65 years or older]

Sex, male, %: [Put as reported, whether by arm or total]

Special Population (e.g., Diagnosis, Comorbidities, Race): [Indicate here any specific populations of interest that the study targets. These populations will be pre- determined within the study protocol. It is expected that all coverage memos will seek to highlight analyses performed by age, gender, or race. Additional populations of interest will be defined by clinical domain or intervention.]

Outcome 1: [Primary outcome as defined by the CAG to be abstracted during the environmental scan. The summary here should present these by arm and indicate if there is a statistically significant difference between arms.] [Report outcomes for any subpopulation analyses.] [Note the follow-up time reported since many studies report data for outcomes at different timepoints, even within the same study.]

Outcome 2:

Outcome 5:

SAMPLE DATA ABSTRACTION

Reference: Burmester et al. 201110

Country: USA

Study Design: Single center RCT

Purpose: To determine whether initial warfarin dosing based on genetic and clinical parameters was superior to an algorithm using only clinical parameters in predicting stable therapeutic dose of warfarin and in anticoagulation outcomes

Study Quality1: Poor Investigators and study staff blinded, patients not blinded, adequate randomization, unclear ITT analysis, >10% difference in attrition between groups, baseline characteristics similar between groups

Funding Source: AHRQ (grant 1RO1HS016335-01), Third Wave Technologies, Osmetech Molecular Diagnostics

Notes: Trial terminated early after interim power analysis showed very low power for observing a significant difference with respect to time in therapeutic range, even if the targeted enrollment (n=130 patients per arm) was pursued Intervention: PG + clinical algorithm guided Coumadin dosing Algorithm components: Sex, age, need for valve replacement, BSA Alleles included: CYP2C9 (*2, *3), VKORC1 (position-1639), CYP4F2 (rs2108622)

Comparator: Clinical algorithm guided Coumadin dosing

Length of follow-up, mean: NR Follow-up period: 60 days (80% of patients completed entire follow-up period)

Patients (N): 230 Intervention: 115 Comparator: 115

Age, years, median years (range):

PG: 67.4 (40.4-89.6), Clinical: 69.2 (40.0-90.6) Age ≥ 65, n (%): NR

Sex, male, %: PG: 57, Clinical: 61

Diagnosis (indication for anticoagulation), PG vs. Clinical, %:

Arrhythmia: 43 vs. 49 Thromboembolic disease: 38 vs. 38 Valve surgery: 21 vs. 17

Comorbidities, n (%): NR

Inclusion criteria: Patients new to warfarin therapy or restarting therapy without a documented stable warfarin dose, patients of white ancestry (including Hispanic white), ≥40 years old, target INR 2-3.5

Exclusion criteria: Pregnancy; known Native American, African American, or Asian descent; thrombocytopenia; severe to moderate hepatic insufficiency; other clinical contraindications for warfarin as deemed by patient’s physician Major hemorrhage, PG vs. Clinical, n (%):

Per DSMB criteria: 3 (2.61) vs. 4 (3.48) Graded as life-threatening or fatal: 2 (1.74) vs. 1 (0.870)

Minor hemorrhage, PG vs. Clinical, n (%):

97 (84.3) vs. 83 (72.2)

Note: AEs that met DSMB criteria defined as serious events that occurred during the trial including all deaths and other unanticipated health events, particularly thromboembolic events and serious hemorrhagic events. Life-threatening hemorrhage defined as bleeding that has led to cardiopulmonary arrest, surgical or angiographic, intervention, or irreversible sequelae.

Related to primary indication for anticoagulation: NR

Other thromboembolic event, PG vs. Clinical, n (%):

Per DSMB criteria: 3 (2.61) vs. 1 (0.870) Graded as life-threatening or fatal: 0 (0) vs. 6 (5.22)

Note: Life-threatening thromboembolic event defined as causing irreversible damage, requires an emergency procedure, or requires admission to an intensive care unit.

Mortality, PG vs. Clinical, n (%): 2 (1.74) vs. 3 (2.61)

Note: No deaths were deemed study related.

1Assessed using the U.S. Preventive Services Task Force (USPSTF) Quality Assessment of Individual Studies tool for randomized controlled trials <https://uspreventiveservicestaskforce.org/uspstf/about-uspstf/methods-and-processes/procedure-manual/procedure-manual-appendix-vi-criteria-assessing-internal-validity-individual-studies> Only significant results are designated by an asterisk: *p<0.05, **p<0.01, ***p<0.001 Abbreviations: AE: adverse event; BSA: body surface area; NA: not applicable; NR: not reported; PG: pharmacogenetic; RCT: randomized controlled trial; [insert other abbreviations as needed

Appendix C. Excluded Studies Table C1. Excluded Studies

Appendix D. Evidence Summary Table D1. Evidence Summary of Intervention XXX versus Intervention YYY.

(Sample Size)

(Rationale)

Efficacy

Intermediate or Final Outcomes

(total number of patients across studies)

(Criteria for downgrading SOE is described; when these criteria are insufficient for understanding the final SOE, additional explanation is provided)

Intermediate or final outcome of interest 2

Intermediate or final outcome of interest 3

Intermediate or final outcome of interest 4

outcome 1

Adverse event outcome 3

Adverse event outcome 4

Appendix E. GRADE Assessment of Quality of Evidence Base

Table E1. Overall Assessment of Quality of Evidence Base Outcome

Follow-up

Outcome

XX RCTs reported in XX papers

STUDY NAME Author Year Author Year

Total:

Follow-up period:

Follow-up period Author Year Outcome: [results]

Author Year Outcome: [results]

Outcome

XX RCTs reported in XX papers

STUDY NAME Author Year Author Year Total:

Follow-up period:

Follow-up period Author Year Outcome: [results]

Author Year Outcome: [results]

References

1. DistillerSR EP. Tool to Assess Risk of Bias in Randomized Controlled Trials. 2022. www.evidencepartners.com/resources/methodological-resources/tool-to-assess-risk-of-bias-in-randomized-controlled-trials-distillersr

2. Appendix VI. Criteria for Assessing Internal Validity of Individual Studies. U.S. Preventive Services Task Force. 2017. www.uspreventiveservicestaskforce.org/uspstf/about-uspstf/methods-and-processes/procedure-manual/procedure-manual-appendix-vi-criteria-assessing-internal-validity-individual-studies

3. Hartmann KE MD, Chang S, . Assessing applicability of medical test studies in systematic reviews. AHRQ. 2022. https://effectivehealthcare.ahrq.gov/sites/default/files/pdf/methods-guidance-tests-applicability_methods.pdf

4. Siemieniuk, R. and Guyatt, G. GRADE (Grading of Recommendations, Assessment, Development and Evaluations) is a transparent framework for developing and presenting summaries of evidence and provides a systematic approach for making clinical practice recommendations. BMJ Publishing Group. 2022 https://bestpractice.bmj.com/info/us/toolkit/learn-ebm/what-is-grade/

Revision History The original version of this Methods Guide was developed as a collaboration between CMS and AHRQ, with contributions by a technology assessment expert (Gillian Sanders Schmidler, PhD) from Duke University (finalized September, 2020). The document was subsequently revised in September, 2022 to reflect pragmatic experience with the guide when applied to 16 separate topics, two of which were technology assessments. Two evidence review report templates were also revised along with the Methods Guide, including the New Technology Add-on Payment and Evidence Preview templates. The revised draft was then reviewed by an external technology assessment consultant (Terry Rogstad, MPH), who suggested further revisions.

image1.jpeg