Attachment_5_-_Surveillance_Procedures_Guide.pdf

Surveillance for the Need to Update:

A Step-by-Step Guide For Comparative Effectiveness Reviews

Agency for Healthcare Research and Quality ECRI Institute

Southern California Evidence-based Practice Center University of Ottawa Evidence-based Practice Center

Table of Contents

Project Management

Create a matrix for project and topic management Identifying Topic Lead Investigators

Appointing Topic Lead Investigators Creating and Sending Questionnaire Matrices

1. Creating the Questionnaire Matrix for the Original Project Lead

2. Sending the CER Project Lead the Questionnaire Matrix and Informational Letters

3. Creating the questionnaire matrix and informational letter for the original CER-spécifique experts, peer reviewers, and/or the local content experts

4. Sending the Questionnaire Matrix to the Original Experts, Peer Reviewers, and/or the Local Content Experts

Library Searches

Abbreviated Library Searches Screening and Data Abstraction

1. Title/Abstract Screening Conducted by the Topic Lead Investigator

2. Full Text Screening

3. Data Abstraction of Included Studies

4. Creation of Summary Table

Input from experts

1. Collating Input from Experts in the Summary Table

ECRI surveillance on the FDA, Health Canada, and MHRA Websites FDA, Health Canada, and MHRA Alerts

Data analysis and evidence synthesis

Writing the Mini-Assessment report

1. Assembling the Report Monthly Conference Calls and Quarterly Reports

1. Monthly conference calls with the TOO

2. Submission of quarterly reports to the TOO

Completed Topics

Project Management Create a matrix for project and topic management

At the beginning of the project, staff should create a matrix to track the progress and flow for each individual Comparative Effectiveness Review (CER) assessment. The matrix is created by entering all the assigned CER topics into Excel or Microsoft Project in the left-most column and listing out the steps involved across the top. Project staff can then enter the date each step was completed for each CER topic. Once a month, the EPC should email Jenae Benns at AHRQ to query which CERs have been released, which are delayed, etc.

Timeline:

Update the flowchart for every team meeting so that project team knows the progress for every given topic. Team meetings are scheduled bi-weekly.

Resource:

Matrix template (attached)

Identifying Topic Lead Investigators Appointing Topic Lead Investigators

The EPC charged with conducting the surveillance assessments appoints a Topic Lead Investigator for each assigned CER. The criteria for choosing this person can be based on the interests and prior work of the investigators associated with the EPC or topic assignments may be made on a rotating basis. The Topic Lead Investigator will be responsible for completing the following steps in the assessment of the CER: drafting and/or reviewing the informational letters addressed to project leads and technical experts; drafting and/or reviewing the questionnaire matrix that contains the conclusions of the original CER (see below); familiarizing him/herself with—and applying—the inclusion/exclusion criteria of the original report; creating a list of the 5 specialty journals to be included in the search (see below); screening titles/abstracts; abstracting the included studies into evidence tables;

collating the responses from the subject matter experts into a summary table; reviewing any relevant US Food and Drug Administration (FDA), Health Canada, or MHRA ([the UK] Medicine and Health Care Products Regulatory Agency) alerts; detecting qualitative and/or quantitative signals (with the assistance of a statistician if needed); and arriving at a decision as to whether the conclusions from the original CER are up-to-date or out-of-date, based on the combination of new evidence and expert opinion.

Timeline:

Prior to 6 months after the date of the CER’s release, CER topics should be assigned to

Topic Lead Investigators.

Summary Table template

Creating and Sending Questionnaire Matrices

1. Creating the Questionnaire Matrix for the Original Project Lead

Six months after a CER is released by AHRQ, it is due for updating surveillance. A questionnaire matrix is created by copying the key questions and conclusions from the original CER’s executive summary. If no concise summary conclusions are provided, the Topic Lead Investigator should summarize the pertinent conclusions to include only conclusions of clinical relevance and to avoid overburdening the subject matter experts. For example, we would not include conclusions such as “rigorous randomized controlled trials are needed.” The matrix that is sent to the original Project Lead includes a column for each of the original technical expert panel (TEP) members:. The informational letter asks the original Project Lead to identify which TEP member(s) would be best able to assess the currency of each conclusion (which usually depends on the TEP member’s area of expertise.)

Before the matrix is sent out, the Topic Lead Investigator reviews the matrix and informational letter.

Timeline:

Staff can create the questionnaire matrices anytime after the CER is released by AHRQ.

Sample Project Lead matrix (attached)

2. Sending the CER Project Lead the Questionnaire Matrix and Informational Letters

When the CER topic is due for assessment, an informational letter is sent to the original CER Project Lead along with the matrix just described and an AHRQ conflict-of-interest disclosure form.

Timeline:

We ask the CER Project Lead to respond within two weeks and send reminders if necessary.

Sample Project Lead informational letter (attached)

3. Creating the questionnaire matrix and informational letter for the original CER-subject matter experts, peer reviewers, and/or the local content experts

Once the Topic Lead Investigator reviews the original Project Lead’s completed matrix, the matrix is modified for the experts (those TEP members or peer reviewers identified by the original Project Lead), replacing the subject matter experts’ names with a set of 3 alternative choices for the expert to indicate, for each conclusion in the original report, whether to his/her knowledge, the conclusion remains up to date or is out of date, or s/he doesn’t know..

The matrix provides space for the expert to add references for any new studies that may affect a conclusion as well as space at the end to provide any additional, related information that does not address a specific key question or conclusion.

An informational letter that includes instructions on how to complete and submit the matrix accompanies the matrix. The letter states the purpose of the surveillance, provides a brief overview of the process, and describes the task we’re asking them to perform and the timeframe; in particular, we emphasize that we tell them we are not asking them to run a new literature search. The letter also states that an honorarium will be provided to each subject matter expert who provides a response.

Timeline:

After approval by the Topic Lead Investigator

Resources:

Sample expert matrix (attached)

Sample informational letter (attached)

4. Sending the Questionnaire Matrix to the Original Experts, Peer Reviewers, and/or the Local Content Experts

Once the original Project Lead returns his or her questionnaire matrix and the Topic Lead Investigator reviews it, informational letters, expert matrices, and conflict of interest disclosure forms are sent out to the experts and/or peer reviewers identified by the Project Lead.

Timeline:

We ask the experts and/or peer reviewers to return the completed matrix (or to tell us if they are unable to comply) within two weeks. Reminders are sent if necessary.

Library Searches Abbreviated Library Searches

When a CER is due for periodic surveillance, the project assistant (or whoever is managing the timeline) emails the Topic Lead Investigator assigned to that CER and the Librarian who will conduct the searches to inform them that the topic is due for an assessment and that the searches should be conducted. According to the abbreviated search protocol, the search includes five leading general medical interest journals (Annals of Internal Medicine, British Medical Journal, Journal of the American Medical Association, Lancet, and the New England Journal of Medicine) and 4 to 6 topic-specific specialty journals. The Topic Lead Investigator will identify and provide the librarian with the names of these additional specialty journals, based on those most highly represented among the references (accepted articles) in the original report. The Librarian reviews the original search strategies and follows them to the extent possible, searching only Medline, either via Pubmed or Medline on OVID, depending on the platform used in the original search, setting the beginning date for the searches six to twelve months prior to the last search conducted by the original EPC.

Additional databases may be searched on a case-by-case basis (e.g., if two or more of the specialty journals are not catalogued on Medline). The librarian sends the search output as an EndNote library, along with the search strategy and a document listing the references and abstracts to the project assistant and Topic Lead Investigator.

Timeline:

A couple of days are sufficient to do the searches.

Screening and Data Abstraction

All the records identified by the update search will be assembled in a Reference Manager database such as Endnote or Refman.

1. Title/Abstract Screening Conducted by the Topic Lead Investigator

After the searches are complete, the Topic Lead Investigator will screen the abstracts identified by the update searches using the original inclusion/exclusion criteria from the report (as well as the abstracts for any articles suggested by an expert: see below).. The Project Assistant will check the selected titles against the reference lists in the original report (included and excluded articles), exclude any already included in the original report, and notify the Topic Lead Investigator of titles that were excluded in the original report. S/he will then order the full text for the remaining abstracts were selected for further review.

2. Full Text Screening

The Topic Lead Investigator will review the full text articles and, using the original inclusion/exclusion criteria, s/he will determine if the study should be included as new evidence and abstracted into an evidence table. It is often difficult to determine from the abstract whether an article contains the outcome data needed to address a particular conclusion; thus, some of the accepted abstracts will be excluded upon full-text screening.

The Topic Lead Investigator will record exclusion decisions in a systematic way.

Article tracking template (attached)

3. Data Abstraction of Included Studies

The Topic Lead Investigator will abstract the data from the included studies into a small evidence table that the Project Assistant creates for the topic. This evidence table will be included in the mini-assessment that is prepared for AHRQ. Inclusion in the evidence table is based on the criteria from the original report.

Timeline for screening and abstracting the data:

Average of one to two weeks

Resource:

Evidence Table template (attached)

4. Creation of Summary Table

The Topic Lead Investigator will create a Summary Table by modifying the matrix to create columns for brief summaries of new evidence, summaries of any health agency warnings, expert input (see below), and the final updating decision. The summary table is included in the mini-assessment prepared for AHRQ.

Timeline for Creating Summary Table :

Approximately a week (included in time for data abstraction)

Summary Table Template (attached)

Input from experts

1. Collating Input from Experts in the Summary Table

After receiving responses from 3-5 experts, the Topic Lead Investigator will collate the responses and add this input to the Summary Table. If fewer than 3TEP members and peer reviewers provide responses, the team will seek out other, preferably local, content experts and send them the informational letter and questionnaire matrix.

Summary table template (attached)

ECRI surveillance on the FDA, Health Canada, and MHRA Websites FDA, Health Canada, and MHRA Alerts

The 2011-2012 Surveillance Program worked with the ECRI Institute, who monitored, at monthly intervals, regulatory agency websites on all assigned topics for any new regulatory information or safety alerts about drugs relevant to the included CERs. The steps that they followed are listed here. If there is a change in the topics, or if a new one is added that now needs to be monitored, the Project Assistant contacts ECRI. Any relevant alerts are then incorporated into the summary table by the Topic Lead Investigator.

Step 1: Prepare a list of final CERs to be reviewed for potential updates.

Step 2: Identify regulatory agency Web sites such as the FDA, Health Canada, and the Medicines and Healthcare products Regulatory Agency (MHRA) as sources for periodic alerts and other related regulatory information on drug products and medical devices.

Step 3: Review the Effective Health Care (EHC) Program Web site daily to identify when the status of a CER becomes “final.”

Step 4: Register for daily notifications of newly posted final CERs via the EHC Web site.

Step 5: Register for daily drug- and device-related notifications from each of the above-listed regulatory agencies.

Step 6: Conduct daily monitoring of the Web sites of the above-listed regulatory agencies for drug- and device-related notifications.

Step 7: Prepare a spreadsheet to capture all information that might suggest the need for the identified CERs to be updated.

o Retrieved information may include drug or product information, which EHC report is likely to be affected, link to the EHC report, source of notification, type of notification, content of notification, link to notification, EPC comments/rationale on which sections of the report might need to be updated, and any ongoing safety reviews.

Step 8: The EPC performing surveillance generates and narratively submits monthly alert summary reports of all pertinent abstracted information from these Web sites that might necessitate updates of CERs’ major sections, such as key questions, discussions, and conclusions.

Step 9: Schedule conference calls or email exchanges with collaborating EPCs to coordinate efforts and summarize progress.

Step 10: Prepare a final report describing methods adopted throughout the project’s course.

Data analysis and evidence synthesis

Once the data are assembled from the literature review, experts, and regulatory websites, an assessment is made on a conclusion-by-conclusion basis regarding the presence of qualitative and quantitative signals that the conclusion may be out of date (these signals are based on a method of conducting updating surveillance that was designed and validated by the University of Ottawa EPC and described further in the EPC report, “Surveillance and Identification of Triggers for Updating Systematic Review: Implementation and Early Experience”). This step may include conducting a small meta-analysis using the pooled outcome(s) from the original report and new data that qualify for inclusion, according to the method, After this task is completed, the team makes an assessment of the priority for updating the whole report, based on the strength of new evidence for all of the conclusions. Details of how these assessments are made are in “Surveillance and Identification of Triggers for Updating Systematic Review: Implementation and Early Experience”.

Writing the Mini-Assessment report

1. Assembling the Report

With the collaboration of AHRQ, a template for the mini-assessments was created by the 2011-2012 Surveillance Program. Using the template, the Topic Lead Investigator and the Project Assistant complete the assessment by inserting the relevant information for the particular CER topic. The Topic Lead Investigator will provide the literature flow and key findings, , complete the summary and evidence tables, and provide a summary decision (either “Low,” “Medium,” or “High” priority). The Project Assistant will insert the title and release date of the original CER, the surveillance date, a list of the assessment authors, and a list of participating subject matter experts who agreed to be identified in the assessment. The Project Assistant and Topic Lead Investigator will tailor the information in the Introduction, Methods, and Results as needed for the particular topic (e.g., if the search methodology deviated from the usual protocol). Lastly, the Project Assistant will insert the summary and evidence tables (if not already done by the Topic Lead Investigator) and run the references using the AHRQ reference style. Once the assessment is ready, the EPC Director will do a final review before the Project Assistant submits the report to the AHRQ Task Order Officer

(TOO).

Timeline for screening and abstracting the data:

Timeline for the whole assessment is three months

Mini-assessment template (attached)

Monthly Conference Calls and Quarterly Reports

1. Monthly conference calls with the TOO The team will have a monthly conference call with the TOO to discuss the progress of the project as a whole. A progress report is sent out with the meeting reminder and is reviewed during the call.

2. Submission of quarterly reports to the TOO Quarterly reports are sent to the TOO, outlining which topics are in progress, which are completed, and what milestones have been reached for individual projects.

Completed Topics

The table below lists the topics in progress of the Surveillance Program and completed topics as well as the topics that the Surveillance Program could not undertake because the CERs were not released in time to start the surveillance assessment.

CER Topic Release date Submission of assessments

Priority Start of the next surveillance

Comparative Effectiveness of Management Strategies for Renal Artery Stenosis: 2007 Update (CER 5)

11/19/07 1st assessment:

8/28/12

Medium 2/28/13

Comparative Effectiveness of Percutaneous Coronary Interventions and Coronary Artery Bypass Grafting for Coronary Artery Disease

(CER 9)

10/1/07 1st assessment:

In Progress

Comparative Effectiveness of Therapies for Clinically Localized Prostate Cancer (CER 13)

2/1/08 1st assessment:

5/18/12

High No re-assessment as this was classified with a high priority

Comparative Effectiveness of Radiofrequency Catheter Ablation for Atrial Fibrillation (CER 15)

7/6/09 1st assessment:

11/30/11

2nd assessment:

12/12/12

Low 6/12/13

Comparative Effectiveness of Lipid-Modifying Agents (CER 16) 9/1/09 1st assessment:

12/23/11

High No re-assessment as this was classified with a high priority

Comparative Effectiveness of Medications To Reduce Risk of Primary Breast Cancer in Women (CER 17)

9/14/09 1st assessment:

11/08/11

2nd assessment:

8/17/12

Medium 2/17/13

Comparative Effectiveness of Angiotensin Converting Enzyme Inhibitors or Angiotensin II Receptor Blockers Added to Standard Medical Therapy for Treating Stable Ischemic Heart Disease (CER 18)

10/16/09 1st assessment:

12/23/11

2nd assessment:

11/1/12

Low 5/1/13

Comparative Effectiveness of Core Needle Biopsy and Open Surgical Biopsy for Diagnosis of Breast Lesions (CER 19)

12/15/09 1st assessment:

12/16/11

2nd assessment:

8/17/12

Medium 2/17/13

Comparative Effectiveness and Safety of Radiotherapy Treatments for Head and Neck Cancer (CER 20)

5/27/10 1st assessment:

11/4/11

Medium 2/31/13

Priority Start of the next surveillance

2nd assessment:

8/31/12

Comparative Effectiveness of In-Hospital Use of Recombinant Factor VIIa for Off-Label Indications vs. Usual Care (CER 21)

6/1/10 1st assessment:

2/24/12 2nd assessment:

11/16/12

Low 5/16/13

Comparative Effectiveness of Nonoperative and Operative Treatments for Rotator Cuff Tears (CER 22)

-7/5/10 1st assessment:

2/24/12

2nd assessment:

11/30/12

Low 5/30/12

Effectiveness of Recombinant Human Growth Hormone (rhGH) in the Treatment of Patients With Cystic Fibrosis (CER 23)

10/4/10 1st assessment:

12/16/11

2nd assessment:

10/5/12

Low 4/5/13

Traumatic Brain Injury and Depression (CER 25) 4/13/11 1st assessment:

4/10/12

2nd assessment:

11/30/12

Low 5/30/12

Comparative Effectiveness of Therapies for Children with Autism Spectrum Disorders (CER 26)

4/4/11 1st assessment:

01/23/12

2nd assessment:

10/31/12

1st: Low

2nd:

Medium

4/31/13

Effectiveness of Disease-modifying Anti-rheumatic drugs (DMARDs) in Children with Juvenile Idiopathic Arthritis (JIA) (CER 28)

9/26/11 1st assessment:

Comparative Effectiveness of Pain Management Interventions for Hip Fracture (CER 30)

5/17/11 1st assessment:

4/10/12

2nd assessment:

10/24/12

Low 4/24/13

Effectiveness of Early Diagnosis, Prevention, and Treatment of Clostridium difficile-Associated Disease (CER 31)

12/19/11 1st assessment:

10/31/12

Low 4/31/13

Priority Start of the next surveillance

Comparative Effectiveness Review of the Diagnosis and Treatment of Obstructive Sleep Apnea in Adults (CER 32)

8/5/11 1st assessment:

7/31/12

Medium 1/31/13

Nonpharmacologic Interventions for Treatment-Resistant Depression in Adults (CER 33)

9/23/11 1st assessment:

8/17/12

Low 2/17/13

Comparative Effectiveness of Terbutaline Pump for the Prevention of Preterm Birth (CER 35)

9/29/11 1st assessment:

5/4/12

2nd assessment:

In progress

Low

Diagnosis and Comparative Effectiveness of Treatments for Urinary Incontinence in Adult Women (CER 36)

4/9/12 1st assessment:

Screening for and Management of Chronic Kidney Disease Stages

1-3 (CER 37)

1/31/12

1st assessment:

10/31/12

Low 4/31/13

Comparative Effectiveness of First and Second Generation Antipsychotics in the Pediatric and Young Adult Populations (CER 39)

2/20/12 1st assessment:

11/15/12

Low 5/15/13

Effectiveness and Safety of Antiepileptic Medications in Patients with Epilepsy (CER 40)

12/1/12 1st assessment:

11/30/12

High No re-assessment as this was classified with a high priority

Comparative Effectiveness of Treatments for Chronic Pelvic Pain in Women (CER 41)

10/31/12

Low 4/31/13

Comparative Effectiveness of Adjunctive Devices to Remove Thrombus and/or Protect from Distal Embolization in ACS Patients Undergoing PCI of Native Vessels (CER 42)

12/29/11 1st assessment:

Attention Deficit Hyperactivity Disorder: Effectiveness of Treatment in At-risk Preschoolers; Long-term Effectiveness in all Ages; and Variability in Prevalence, Diagnosis, and Treatment (CER 44)

10/21/11 1st assessment:

8/17/12

Low 2/17/13

Comparative Effectiveness of Self-measured Blood Pressure Monitoring (CER 45)

11/14/12

Low 5/14/13

Noninvasive Diagnostic Tests for Breast Abnormalities: Update of a 2006 Review (CER 47)

2/1/12 1st assessment:

Comparative Effectiveness of Hematopoietic Stem-cell Transplantation in the Pediatric Population (CER 48)

2/1/12 1st assessment:

10/29/12

Low 4/29/13

Comparative Effectiveness of Venous Thromboembolism (VTE) Prophylaxis in Orthopedic Surgery (CER 49)

3/13/12

12/14/12

Low 6/14/13

Priority Start of the next surveillance

Effectiveness of Antinuclear Antibody, Rheumatoid Factor, and Cyclic-Citrullinated Peptide Testing for the Evaluation of Musculoskeletal Complaints in Pediatric Populations (CER 50)

3/7/12 1st assessment:

Dietary Supplements in the Elderly (CER 51) 4/30/12 1st assessment:

Drug Therapy for Psoriatic Arthritis in Adults: Update of a 2007 Report (CER 54)

4/1/12 1st assessment:

Drug Therapy for Rheumatoid Arthritis in Adults: An Update (CER 55)

4/1/12 1st assessment:

Pharmacologic Therapy vs Medical Foods for the Treatment of Phenylketonuria (PKU) (CER 56)

2/22/12 1st assessment:

Comparative Effectiveness of Screening for Glaucoma (CER 59) 4/12/12 1st assessment:

Comparative Effectiveness of Treatment for Open-Angle Glaucoma

(CER 60)

4/12/12 1st assessment:

Comparative Effectiveness of Treatment for Depression after Unsatisfactory Response to SSRIs (CER 62)

4/26/12 1st assessment:

Progestogens for Prevention of Preterm Birth 9/7/12 Report not released in time

Comparative Effectiveness of First and Second Generation Antipsychotics in the Adult Population

8/14/12 Report not released in

Comparative Effectiveness of Pharmacologic Therapies for the Management of Crohn’s Disease

Not released Report not released in

Comparative Effectiveness of Allergen-specific Immunotherapy for the Treatment of Allergic Rhinoconjunctivitis and/or Asthma

Comparative Effectiveness of Bariatric Surgery and Non-Surgical Therapy in Adults with Metabolic Conditions and Body Mass Index of 30 to 34.9

Comparative Effectiveness of Breathing Exercises and/or Retraining Techniques in the Treatment of Asthma

9/10/12 Report not released in

Comparative Effectiveness of Procalcitonin-Guided Decisions for the Diagnosis and Management of Sepsis

10/5/12 Report not released in

Comparative Effectiveness and Safety of Multiple Daily Injections or Insulin Pump Therapy With or Without Continuous Glucose Monitoring for Diabetes Mellitus

7/10/12 Report not released in

Priority Start of the next surveillance

Comparative Effectiveness of Surgical Options for Inguinal Hernia 8/17/12 Report not released in