Attachment_3_-_EPC_Procedures_Guide_(v14).pdf

EPC Procedures Guide For Effective Healthcare Program

Evidence Reviews

Center for Outcomes and Evidence

Agency for Healthcare Research and Quality

EPC Procedure Guide

Please refer to the Secure Site under Resources/EPC Process Resources to identify current deliverable templates

Table of Contents

Useful websites and email addresses 3

Topic Development 4

Sample Schedule of Deliverables 10

Topic Refinement and Surveillance 12

Systematic Reviews and Updates 25

Sample Schedule of Deliverables for Systematic Review 46

Sample Schedule of Deliverables for combined Systematic Review/

Topic refinement 50

Sample Schedule of Deliverables for Surveillance 55

Technical Briefs 56

Sample Schedule of Deliverables 72

Methods and EPC Methods Guidance Reports 71

Sample Schedule of Deliverables 74

Version 14 – 4/15/14

Useful websites and email addresses

Websites

AHRQ www.ahrq.gov

Effective Healthcare Program www.effectivehealthcare.ahrq.gov

PROSPERO http://www.crd.york.ac.uk/NIHR_PROSPERO/

Scholar One http://mc.manuscriptcentral.com/ehc

Secure Site http://www.epc-src.org/src/logon.cfm

Systematic Review Data Repository (SRDR) www.srdr.ahrq.gov

Email Addresses

Eisenberg Center EisenbergCenter@bcm.edu*

EPC coordinator EPC@ahrq.gov

Methods workgroups methods@epc-src.org

ScholarOne questions review@epc-src.org

Secure Site

SIPs and Federal Registry notices

Edwin.Reid@va.gov sips@epc-src.org

Topic Triage

Web-based Disclosure Form and Acknowledgement and Confidentiality Form christel.villarivera@Lewin.com

Edwin.Reid@va.gov

Links

EPC Methods Guide for Effectiveness and Comparative Effectiveness Reviews http://www.effectivehealthcare.ahrq.gov/ehc/products/60/318/CER-methods-guide-130916.pdf

EPC Methods Guide for Medical Test Reviews http://www.effectivehealthcare.ahrq.gov/ehc/products/246/558/Methods-Guide-for-Medical- Test-Reviews_Full-Guide_20120530.pdf

**Will be updated when new website is available http://www.ahrq.gov/ http://www.effectivehealthcare.ahrq.gov/ http://www.crd.york.ac.uk/NIHR_PROSPERO/ http://mc.manuscriptcentral.com/ehc http://www.epc-src.org/src/logon.cfm http://www.srdr.ahrq.gov/ mailto:EisenbergCenter@bcm.edu mailto:EPC@ahrq.gov mailto:methods@epc-src.org mailto:review@epc-src.org mailto:Edwin.Reid@va.gov mailto:sips@epc-src.org mailto:christel.villarivera@Lewin.com

Topic Development Overview:

The goal of Topic Development to develop and select topics for review. The Topic Development team will identify the clinical and/or policy context surrounding the topic nomination, evaluate its fit established selection criteria, and synthesize this information in the Topic Brief. The Topic Brief provides the necessary background information for the Topic Triage discussion and vote.

For the EHC Program, Topic selection criteria are: 1.Appropriateness, 2. Importance, 3.

Duplication, 4. Feasibility, and 5. Potential impact. The Topic Development team presents developed topics to the Topic Triage Group. The Topic Triage group votes on each topic and their disposition. Topics selected to go forward for review are further refined to produce focused key questions (described under Topic Refinement activities).

At the start of Topic Development, the EPC will receive a completed Topic Nomination Form and any supporting documents submitted by the nominator.

Items specified for delivery are reviewed by the TOO before final acceptance. For the EPC Program, de-identified Topic Briefs will be posted publicly on the EHC website. The EPC shall be required to make revisions as deemed necessary by the TOO to ensure a topic brief of high quality. The EPC shall submit deliverables to the Secure Site, in the format and within the timeframes indicated, and notify the specified recipient (generally the TOO) of the upload.

The TOO may request that the EPC also email a copy of the deliverable to the TOO. Due dates which fall on a weekend or holiday are due on the next business day.

Resource

• Documents describing the Topic Development process (including templates and examples) on the Secure Site at: Resources /EPC Process Resources /04. Topic Development.

• Training on topic development (in Power Point) presented by the Scientific Resource Center (SRC) at the November 2009 EPC Directors meeting. On the Secure Site at:

Resources /EPC Process Resources /04. Topic Development.

The Topic Development Process and Deliverables:

1. Staffing plan and Project management file

The Staffing plan identifies key staff working on the project, and the Project management file lays out the timeline for the project and specific deliverables. Personnel can include a principal investigator who oversees the scientific content of topic development, a project director who is responsible for management of Topic Development, a research associate who conducts the research and helps prepare the topic briefs, a research librarian, and clinical experts who provide clinical feedback.

Deliverable to AHRQ

• Completed Staffing Plan. This should list all project personnel, their roles, and the primary point(s) of contact for the project. Upload to Secure Site and notify TOO.

• Project management file with each deliverable and a deliverable date. Upload to Secure Site and notify TOO.

Resource

• Staffing plan template on the Secure Site at: Resources /EPC Process Resources /01.

EPC Program Policies and Procedures /Work Files and Contract Process Forms

2. Staff Disclosure Form, and Acknowledgement and Confidentiality Agreement

The EPC shall upload a signed Disclosure form and Confidentiality Agreement for each individual who will assist in the Topic Development process including core staff. EPCs may choose to use web-based disclosure forms and confidentiality agreements. The TOO will review and approve all proposed staff before their participation.

The EPC should receive acknowledgment of the receipt and acceptance from the TOO of the Disclosure forms, and if relevant the plan for management of any conflicts of interest. If the EPC does not receive this within a week, follow-up should ensue on the part of the EPC.

The EPC shall maintain confidentiality on the findings of the topic development. The EPC is not to present or publish any findings until the Topic Nomination Disposition Summary is posted.

EPC’s can share what is publically available. The program has specific venues for solicited expert input through engagement of specific stakeholders or experts. The EPC shall refer any unsolicited inquiries or comments outside of these structured processes to the TOO.

Deliverable to AHRQ

• Signed Disclosure forms. Upload to Secure Site and notify TOO.

• Signed Confidentiality Agreements. Upload to Secure Site and notify TOO.

Resource

• AHRQ Evidence-based Practice Center Policy on Financial and Non-financial Interests, and Disclosure Form for Conflict of Interest (COI) on Secure Site at:

Resources /EPC Process Resources /01. EPC Program Policies and Procedures

• EPC Review Team Confidentiality Agreement on Secure Site at: Resources /EPC Process Resources /01. EPC Program Policies and Procedures

• Web-based Disclosure Form and Confidentiality Agreement. Contact the SRC for information and access.

3. Targeted Topic Development Discussions with Specific Stakeholders or Experts

The purpose of these discussions is to clarify the contextual issues driving the nomination.

Discussions are to be limited and efficient, focused solely on providing initial direction in developing the topic to be responsive to the decisional needs of stakeholders. Input can be garnered from local clinical experts (including primary care providers and sub specialists), the nominator, or other stakeholders readily available to the EPC. Since discussions are limited and for the purpose of background, Disclosure of Interest forms are not needed from these individuals. The EPC uses this input to understand aspects of the topic such as practice variation, clinical uncertainty, appropriate comparators and important subpopulations.

Discussions with the nominator may be helpful for clarification, context, and to understand specific requirements (for example, if the nominator indicates that they are requesting a review to support guideline development, it is helpful to know when they hope to issue the guideline).

Deliverable to AHRQ

• A summary of the discussions should be documented on the topic brief cover sheet (topic brief described below).

Resource

• Sample topic briefs on Secure Site at: Resources /EPC Process Resources /04. Topic Development /Topic Nomination Development Resources & Templates /Example Documents

4. Development of the Topic Brief

The EPC completes a Topic Brief for each nomination as a part of Topic Development. The Topic Brief informs the topic selection discussion.

The three parts of the Topic Brief are:

• Topic Triage Cover Sheet. The purpose of the cover sheet is to summarize the nomination and the key considerations supporting the EPC staff recommendation as to whether a topic should go forward as a research review. The Cover Sheet should therefore include:

o Documentation of targeted discussions with specific stakeholders or local topic experts held to clarify the topic nomination and provide direction on the refinement of the topic.

o An outline of key considerations and discussion points for the Topic Triage selection discussion.

o A Draft/Preliminary key questions document in the Patient, Intervention, Comparison, Outcomes, Timing, and Setting PICO(TS) format. The Timing and Setting sections are only used when applicable o If applicable, a list of 2-3 relevant publications to be reviewed by voting members in order to increase familiarity with the topic (i.e. review articles, not important guidelines or studies unless they are the best way to educate the Topic Triage members on the topic);

o A final staff recommendation, by the EPC, including the disposition code (see reference materials for code sheet)

• Selection Criteria Table. The purpose of this table is to document a nomination’s fit with each of the program’s selection criteria.

These criteria are: 1.Appropriateness, 2. Importance, 3. Duplication, 4. Feasibility, and 5. Potential Impact. Key points from this document can be summarized in the topic triage coversheet.

• Existing Guidance Table. The purpose of this table is to document existing evidence on the topic including reviews, guidelines, policies, and primary research.

Deliverable to AHRQ

• Three-part Topic Brief. Upload to Secure Site and notify TOO. During the EPC’s first experiences with Topic Development, Topic Briefs will be reviewed by the Scientific Resource Center (SRC) and TOO. The SRC will add the topic for presentation at a Topic Triage meeting. Once approved by the TOO, the SRC will upload the documents to the appropriate Topic Triage meeting folder on the Secure Site. They will also send an email to the Topic Triage group alerting them to the availability of the documents on the Secure Site.

Resources

• Templates for the three parts of the Topic Brief (i.e. Topic Triage Cover Sheet, Selection Criteria Table and Existing Guidance Table) on the Secure Site at:

Resources /EPC Process Resources /04. Topic Development /Topic Nomination Development Resources & Templates

• Examples of completed documents on the Secure Site at: Resources /EPC Process Resources /04. Topic Development /Topic Nomination Development Resources & Templates /Example Documents

• Guidance for assigning Topic Triage Coding Recommendations on the Secure Site at:

Resources /EPC Process Resources /04. Topic Development /Topic Nomination Development Resources & Templates /Process Documents

5. Presentation

Topic briefs conducted on behalf of the EHC program or US Preventive Services Task Force (USPSTF) may be required to present the findings of the Topic Brief. The purpose of this presentation is to summarize the Topic Brief document and answer questions about the scope of the nomination. Note that this presentation and subsequent steps may be a separate task.

All of the topics developed for the EHC program will be presented to a Topic Triage Group by the EPC. The EPC may assume that the Topic Triage members have reviewed the Topic Brief in advance of the meeting. The Topic Triage Group, comprised of members from AHRQ staff, the SRC, and the Eisenberg Center (EC) meets at least once per month (typically the first Wednesday of the month between 12-2 pm EST) to review, discuss, and vote whether a given topic nomination, informed by the Topic Brief, will go forward for further product development in the EHC program (as a technical brief, update report, or topic refinement with systematic review).

Deliverable to AHRQ

• A 2-3 minute presentation. The EPC should also be available to answer any questions during the group discussion. For the EHC Program, the EPC should contact the TOO or christel.villarivera@Lewin.com for dial-in information.

6. Consultation on Nomination Summary Document

The EHC program posts a Nomination Summary Document on the EHC website to inform the public of the outcome of each topic nomination.

After the Topic Triage meeting, the SRC will prepare minutes of the Topic Triage meetings and mailto:christel.villarivera@Lewin.com finalize the Topic Triage Coversheet. The SRC will also create a Nomination Summary Document that outlines the disposition for each triaged nomination. For topic briefs conducted for the EHC program, the EPC will be expected to review the Nomination Summary Document(s) prior to posting. Finalized cover sheets (with the disposition and meeting minutes incorporated) will be posted by the SRC on the Secure Site. The Nomination Summary Document describing the disposition will be posted by AHRQ/EC on the EHC website and will be sent to the nominator by the SRC.

Deliverable/Activity

• Review a Nomination Summary Document at the request of the SRC.

Resource

• Sample Nomination Summary Documents on the Secure Site at:

Resources /Topic Triage /NSDs already posted

7. Responding to feedback generated by Nomination Summary Document

The purpose of responding to feedback generated by Nomination Summary Documents is to ensure that the EPC program is responsive to the public and that all decisions are transparent.

If a nominator or member of the public sends a comment or question concerning a Nomination Summary Document, the SRC may ask the EPC to draft a response because of their knowledge and familiarity with the topic. The response will then be reviewed by the SRC and the TOO before the SRC sends the response to the person who submitted the comment/question.

Deliverable/Activity

• Draft response at the request of the SRC.

Resource

• Feedback and Response –Nomination Summary Documents on Secure Site at:

Resources /Topic Triage /Feedback & Responses - Nomination Summary Docs

Topic Development

Sample Schedule of Deliverables Date of Assignment: 1/1/2014 Task Format Work Interval from Award Date Date Due

Pre-Kickoff Call

Submit timeline and staffing plan Electronic 1 week 1/8/2014

Submit Staff Disclosure forms and acknowledgement and confidentiality forms

Electronic 2 weeks 1/15/2014

Kick-off Call Hold call with TOO and EC to discuss goals/objectives of project, key questions, etc.

2 weeks 1/15/2014

Submit summary of call's discussion decisions, to call participants 1 week from kick-off call.

Electronic 1 weeks after kick-off call 1/22/2014

Topic Brief Targeted Topic Development Discussions with Specific Stakeholders or Experts

Prior to submission of draft

Submission of draft Topic Brief to TOO Electronic

2 weeks before Topic Triage meeting 2/15/2014

Submit final Topic Brief to SRC Electronic 1 week before Topic Triage meeting 2/19/2014

Presentation to Topic Triage 8 or more weeks 2/26/2014

Topic Triage

Consultation on Nomination Summary Document 2 weeks after

Topic Triage 3/12/2014

Topic Refinement and Surveillance

The purpose of Topic Refinement is to refine the scope and to tighten up the key questions and PICOTS for Systematic Reviews so they are informative and reflect stakeholder decisional dilemmas. Technical Briefs do not undergo Topic Refinement. The purpose of surveillance is to determine the need for updating an existing review as well as the scope and questions of the update.

The Topic Refinement Document has three parts:

• Topic Refinement Document, Part 1: Development of the Preliminary Scope (Key Questions (KQ), PICOTS and Analytic Framework (AF)) o In initial topic refinement, the EPC prepares the topic scope for discussion with key informants. Part 1 of the Topic Refinement Document should expand the background and clinical context, clarify the preliminary key questions, outline the quantity, quality, and challenges of the literature base, and describe key issues for Key Informant discussions.

• Topic Refinement Document, Part 2: Development of the Draft Scope (KQ, PICOTS and AF) o This summarizes key informant input; an updated literature scan (if undertaken); changes to the KQ, AF and/or PICOTS; and the rationale for these changes. This also includes a list of known pharmaceutical or device companies or other professional entities or researchers from whom Scientific Information Packets (SIP) should be requested once the key questions are finalized (i.e. when the protocol is posted)

• Topic Refinement Document, Part 3: Key Question Posting Document o This includes the draft key questions, draft PICOTS, and draft AF after discussion with the Key Informants. This is posted on the Effective Healthcare website for three weeks for public comment For Topic Refinements developed under the Technology Assessment Program, the draft key questions are posted for two weeks.

Items specified for delivery are reviewed by the TOO before final acceptance. The EPC shall be required to make revisions as deemed necessary by the TOO to ensure a Key Question Posting Document (Topic Refinement Document Part 3) of high quality. The EPC shall submit deliverables to the Secure Site, in the format and within the timeframes indicated. Due dates which fall on a weekend or holiday are due on the next business day.

Resource

• Documents describing the Topic Refinement process can be found on the EHC Secure Site at: Resources /EPC Process Resources /05. Topic Refinement

The Topic Refinement Process and Surveillance Process

1. Staffing plan and Project management file

The Staffing plan identifies key staff, and Project management file lays out the timeline for the project and specific deliverables.

Deliverable to AHRQ

• A completed Staffing Plan. This should list all project personnel, their roles, and the primary point(s) of contact for the project. Upload to the Secure Site and notify TOO

• Project management file with each deliverable and a deliverable date. Upload to the Secure Site and notify TOO

Resource

• Staffing plan template on Secure Site at: Resources /EPC Process Resources /01. EPC Program Policies and Procedures /Work Files and Contract Process Forms

2. Staff Disclosure Forms, and Acknowledgement and Confidentiality Agreements

The EPC shall upload a signed Disclosure form and Confidentiality agreement for each individual who will assist in Topic Refinement (this includes core EPC staff). EPCs may choose to use web-based disclosure forms and confidentiality agreements. Staff members who have already submitted Disclosure forms for the Topic Development phase do not need to resubmit disclosures unless they have acquired new interests or more than a year has elapsed since the previous disclosure. The TOO will review forms and approve all proposed staff before their participation.

The EPC Director is responsible for assembling the team who will work on Topic Refinement and Systematic Review. The Director should review the Disclosure forms for potential financial or non-financial conflicts of interest. For any potential conflicts of interest, the EPC Director may prepare a document describing the potential conflict and any plans in place to manage the potential conflict of interest. The EPC will upload this document to the EHC secure site and notify the TOO. The TOO will request a call with the EPC director if further clarification is needed or if the TOO has concerns about the participation of an individual with a disclosed conflicts of interest.

The EPC should receive acknowledgment of the receipt and acceptance from the TOO of the Disclosure forms, and if relevant the plan for management of any conflicts of interest. If the EPC does not receive this within a week, follow-up should ensue on the part of the EPC.

The Eisenberg Center will send Disclosure forms directly to the TOO prior to the kick-off call.

The EPC shall maintain confidentiality on the findings of the topic. The EPC program has specific venues for solicited expert input through engagement of Key Informants, Technical experts, and peer reviewers, as well as solicited public comment on the key questions and draft report. The EPC shall refer any unsolicited inquiries or comments outside of these structured processes to the TOO.

Deliverable to AHRQ

• Upload signed Disclosure forms to the Secure Site. Notify TOO.

• Upload signed Confidentiality Agreements to the Secure Site. Notify TOO.

Resource

• AHRQ Evidence-based Practice Center Policy on Financial and Non-financial Interests, and Disclosure Form for COI on Secure Site at: Resources /EPC Process Resources /01. EPC Program Policies and Procedures

• EPC Review Team Confidentiality Agreement on Secure Site at: Resources /EPC Process Resources /01. EPC Program Policies and Procedures

• Web-based Disclosure Form and Confidentiality Agreement. Contact the SRC for information and access.

3. Key Informant and Technical Expert List

The EPC shall suggest a small number of individuals (usually 6-8) for the Key Informant (KI) panel. The partner and TOO may suggest additional individuals. The KI panel includes patients and caregivers, practicing clinicians, relevant professional and consumer organizations, purchasers of health care, and others with experience in making health care decisions relevant to the topic. KIs are end-users of the report and may not necessarily be content, clinical, or methodological experts. Thus, KIs are distinct from the Technical Expert Panel (TEP) which is constituted to inform the scientific processes of the subsequent research review. Depending on their expertise, some members of the KI panel may go on to serve as members of the TEP. If the contract calls for a combined Topic Refinement/Systematic Review Timeline, the EPC should also develop a preliminary TEP list. Note: For TA reports, the TEP is comprised solely of CMS staff.

The primary role of the KI is to identify the particular questions to be reviewed. The EPC is seeking input from KIs on the populations, interventions, comparisons, or outcomes of particular interest to them. The input from the KIs will assist the EPC in refining Key Questions and the Analytic Framework. Specifically, the KIs may provide input on:

• Outcomes of particular importance to be graded

• Comparisons of interest or uncertainty which users must decide between

• Populations which should be considered distinct from one another when comparing interventions or tests, or specific populations which may have particular controversy or uncertainty

For the Surveillance process, the primary role of Key Informants at this stage is to provide input as to whether the questions or conclusions from the original report may be considered out of date. These experts may include authors, TEP members, or peer reviewers of the original report.

The TEP informs the scientific content of the research review and provides input to the EPCs throughout the evidence review process, particularly the review protocol. Typically composed of 5 to 8 members, Technical Expert Panels usually include one or more clinical providers, methods experts, experts from relevant federal government agencies, relevant professional society and health care purchaser representatives, and other content experts. The size and composition of the Technical Expert Panel are intended to create a balance between content and methodology expertise and the user’s perspective. During the course of developing evidence reports, Technical Expert Panels may assist the EPCs by providing input to:

• focus the literature search by helping to identify search strategies and relevant grey literature.

• identify inclusion/exclusion criteria to evaluate the quality of studies and rate the strength of the overall body of evidence, etc.

• clarify specific technical or clinical issues that may arise.

• identify specific subgroups which may have heterogeneity of treatment effect. The EPC would collect this information from studies to determine the applicability of evidence and perhaps to conduct sensitivity analysis or stratified analyses.

EPCs must be aware of and abide by the Office of Management and Budget (OMB) Paperwork Reduction Act (PRA) (http://www.hhs.gov/ocio/policy/collection/), which prohibits asking the same question of more than 9 persons.

• The 9 person limit does not apply to US federal government employees, or information gathered from within the EPC program. It does apply to individuals from other countries.

• OMB approval would be required to ask the same question of more than 9 individuals such as for survey purposes. This may include:

o Individual structured or semi-structured interviews o Sending out questions via web or email for people to answer specifically.

• The following may not specifically apply:

o Open discussions of specific agenda items from focus group or expert panels, such as would be during an in-person meeting o Free-form discussions with individuals.

o Posting something for comment or inviting individuals to provide comment.

For the EPC program, AHRQ does not dictate how KI, TEP or other stakeholder calls are handled, but suggest specific attention to the OMB PRA rules and requirements if the EPC plans individual calls with structured or semi-structured interview formats. This may be particularly applicable for methods projects or projects where the stakeholder feedback is a necessary piece of the results of the project.

If the nature of the information gathering is more general, such as from recording open discussions, then the EPC should be careful about how this process is described in the EPC report. The information gathered is more of the nature of feedback or input rather than results or consensus advice.

Deliverable to AHRQ

• List of proposed Key Informants and Technical Experts, if relevant. Upload to Secure Site and notify TOO.

4. Kick-Off Call and Summary

The EPC shall convene and lead a conference call with the TOO. The Eisenberg Center (EC) is invited in a listener role. Optional participants include appropriate AHRQ staff or other http://www.hhs.gov/ocio/policy/collection/ individuals involved in Topic Development. The sponsoring partner may also be invited to the call. A sponsoring partner contributes funding for the project. The purpose of the call is to:

• Review elements of work to be performed

• Review list of Key Informants and Technical Experts

• Describe a plan for working with Key Informants

• Outline the points for partner or sponsoring partner input throughout project.

Establish lines of communication between the EPC, TOO, stakeholders, other AHRQ staff, and partners, including professional organizations and the United States Preventive Services Task Force (USPSTF) or other federal partners. The TOO should be copied in all email communications between the EPC and partner organizations. In the rare occasion that the EPC has a call with a partner without the TOO, the EPC should send a written summary to the TOO.

The roles and responsibilities of the sponsoring partner will be outlined prior to award in a “Roles and Responsibilities of the Partner” document. Types of involvement may include participation in kick-off calls, review of deliverables, review of KI list or TEP list, and attendance on monthly calls. AHRQ will be responsible for approval of all deliverables.

Deliverable to AHRQ

• The EPC shall send an electronic summary of the discussion to all call participants and also upload it to the Secure Site within one week of the kick-off call. The EPC may choose to provide this summary to Key Informants before the Key Informant call.

Resource o Topic Refinement Document on Secure Site at: Resources /EPC Process Resources /05. Topic Refinement /Topic Refinement Templates o Facilitation Primer for engaging stakeholders -http://effectivehealthcare.ahrq.gov/tasks/sites/ehc/assets/File/Facilitation_Primer_20 120124.pdf o “Roles and Responsibilities of Partners” document on the Secure Site at Resources /EPC Process Resources /01. EPC Program Policies and Procedures http://effectivehealthcare.ahrq.gov/tasks/sites/ehc/assets/File/Facilitation_Primer_20120124.pdf http://effectivehealthcare.ahrq.gov/tasks/sites/ehc/assets/File/Facilitation_Primer_20120124.pdf https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=0 https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=6906 https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=6907

5. Discussion with topical experts

The EPC may engage topical experts when completing Topic Refinement Document Part 1 and preparing for key informant discussions. Topical experts are easily accessed individuals who provide input on current practice, available interventions, decisional dilemmas, etc. Often these individuals provide clinical context, and insight into the “real-world” situations of stakeholders.

These discussions will inform the development of the preliminary key questions, PICOTS and analytic framework.

6. Key Informants (KI) Disclosure Forms and Confidentiality Agreements

The PI of the project is responsible for assembling the lists of potential KI and TEPs. The EPC is responsible for recruiting KI and obtaining CVs, signed Disclosure forms, and completed Acknowledgement and Confidentiality agreements from KI. EPCs may choose to use web-based disclosure forms and confidentiality agreements. Contact the SRC for information about these forms. CVs should be obtained from all KIs with the exception of patients or similar participants who may not have readily available CVs. The requirement of disclosure in this instance is made in the interest of transparency and to help create a balance of perspectives among the Key Informants.

The PI should review the Disclosure forms for potential financial or non-financial conflicts of interest. While, in general, Key Informants may have conflicts of interest, the PI may prepare a document (in consultation with the EPC director) describing the potential conflict and any plans in place to manage the potential conflict of interest if a declared interest of particular concern is identified. The EPC will submit this document to the TOO for discussion with EPC program director. The TOO will request a call with the EPC director if further clarification is needed or if the TOO has concerns about the participation of an individual with disclosed conflicts of interest.

The EPC should receive acknowledgment of the receipt and acceptance of the Disclosure forms, and if relevant the plan for management of any conflicts of interest from the TOO. If the EPC does not receive this within a week, follow-up should ensue on the part of the EPC.

KIs should be advised that their names and contact information will be shared with the SRC for potential consultation on future work unless they opt out. Key Informants should also be advised that they will be acknowledged in the final systematic review or technical brief.

NOTE: Individuals may not serve as KIs or TEP until their signed Disclosure forms have been approved by the TOO.

Deliverable to AHRQ o Upload the completed Disclosure forms, CVs if applicable, and signed Acknowledgement and Confidentiality Agreement for KIs to the Secure Site prior to Key Informant calls. Notify TOO for review and approval.

Resource o AHRQ Evidence-based Practice Center Policy on Financial and Non-financial Interests, and Form for Disclosure of Interests on Secure Site at: Resources /EPC Process Resources /01. EPC Program Policies and Procedures o KI and TEP Acknowledgement and Confidentiality Agreement on Secure Site at:

Resources /EPC Process Resources /01. EPC Program Policies and Procedures o KI resources on the Secure Site at: Resources /EPC Process Resources /05. Topic Refinement /KI Resources o Web-based Disclosure Form and Confidentiality Agreement. Contact the SRC for information and access.

7. Surveillance Protocol (For Surveillance of an existing EPC SR)

The EPC shall develop a protocol for surveillance for an existing systematic review. This will not be posted on the EHC website.

Deliverable to AHRQ o Surveillance protocol. Upload to the Secure Site and notify TOO.

Resource o Surveillance Update Guide with Attachments on Secure Site at: Resources /EPC Process Resources /01. EPC Program Policies and Procedures /Surveillance for Update o Protocol Content Guidance on Secure Site at: Resources /EPC Process Resources /08. Systematic Review Protocol

8. Background document (Topic Refinement Document, Part 1 for Topic Refinement and Questionnaire Matrix for Surveillance)

In order to have a useful and effective KI call, all must be familiar with the basic facts about the topic and proposed research. Therefore for Topic Refinement of a new topic, the purpose of Part 1 of the Topic Refinement Document is to lay the groundwork for a productive discussion with https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=0 https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=6906 https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=6907 https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=13632 the KIs. This portion of the document may be used to initiate and not limit discussions with the Key Informants.

To complete Part 1 of the Topic Refinement Document, the EPC will systematically scan the scientific evidence in a targeted manner. This literature scan is not intended to be synthesized, but to inform the scope of the topic.

For Surveillance for updating an existing report, the EPC shall create a questionnaire matrix by copying the key questions and conclusions from the original SR’s executive summary. If no concise summary conclusions are provided, the EPC should summarize the pertinent conclusions to include only conclusions of clinical relevance and to avoid overburdening the subject matter experts. For example, the EPC would not include conclusions such as “rigorous randomized controlled trials are needed.” The EPC should check with the authors of the original report on the accuracy of this matrix before sharing with key informants.

o Part 1 of the Topic Refinement Document. Upload to the Secure Site and notify the

TOO.

o Questionnaire Matrix and Informational letter.

Resource o Topic Refinement Content Guidance Document on Secure Site at: Resources /EPC Process Resources /05. Topic Refinement /Topic Refinement Templates o Questionnaire Matrix and Informational Letter templates on Secure site at: Resources /EPC Process Resources /01. EPC Program Policies and Procedures /Surveillance for Update

9. Host Key Informant Conference Call(s)

The EPC shall invite participants, schedule, and lead conference calls to discuss the appropriateness and scope of the Key Questions and PICO (or in the case of Surveillance, the currency of the original Key questions and conclusions). These are open-ended discussions grounded in a review of Part 1 of the Topic Refinement Brief or Questionnaire Matrix, and dialogue between participants should be encouraged. Participants in the call include the TOO, the topic nominator (if appropriate), the selected KIs, and EC staff.

https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=6907

Logistically, several calls maybe needed to both meet the scheduling needs of KIs (or TEP in the case of Surveillance) and to adequately address the topic under refinement. If multiple calls are necessary, the EPC may consider holding the patient or consumer call separately, since patients or consumers may find it difficult to express their views during a call dominated by other seeming experts.

Deliverable to AHRQ o Send minutes of call discussions to call participants, upload a copy to the Secure Site, and notify the TOO.

o A high-level summary of the call(s) will be documented in the Topic Refinement Document Part 2.

Resource o Meeting summary template on Secure Site at: Resources /EPC Process Resources /05.

Topic Refinement /Call Resources o Topic Refinement Content Guidance Document on Secure Site at: Resources /EPC Process Resources /05. Topic Refinement /Topic Refinement Templates o Stakeholder Engagement Modules on Secure Site at: Resources /EPC Process Resources /05. Topic Refinement /Stakeholder Engagement Training o Key Informant and Key Informant Call resources on Secure Site at: Resources /EPC Process Resources /05. Topic Refinement /KI Resources o Surveillance Update Guide with Attachments on Secure Site at: Resources /EPC Process Resources /01. EPC Program Policies and Procedures /Surveillance for Update

10. Final Documents (For Topic Refinement of new topics, the Topic Refinement

Document, Parts 2 and 3. For Surveillance of existing SRs, the Surveillance Report)

Part 2 of the Topic Refinement Document includes a high-level summary of KI discussions;

details of an updated literature scan if done; and rationale for changes to the KQ, PICOTS and/or AF. It also includes a list of known pharmaceutical or device companies, other professional entities, or researchers from whom Scientific Information Packets will be requested.

Part 3 of the Topic Refinement Document is the Key Question Posting Document. Part 3 and the alternative text for all figures will undergo AHRQ internal review. After review and approval it https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=6906 https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=6907 will be posted on the EHC website for public comment.

The EPC shall conduct similar steps for the Surveillance process. The EPC shall incorporate findings from KI discussions, FDA alerts, and an abbreviated search and abstraction of results through qualitative or quantitative means into a surveillance report. The Surveillance Report should clearly address the currency and scope of the original SR key questions, as well as the priority (“low”, “medium”, “high”) for updating each key question. The Report should clearly state the rationale behind the recommendations.

The abbreviated search shall be based on the original search strategy and applied to the five leading general medical interest journals (Annals of Internal Medicine, British Medical Journal, Journal of the American Medical Association, Lancet, and the New England Journal of Medicine) and 4 to 6 topic-specific specialty journals (based on those most highly represented among the references (accepted articles) in the original report). The abbreviated search may search only Medline, either via Pubmed or Medline on OVID, depending on the platform used in the original search, setting the beginning date for the searches six to twelve months prior to the last search conducted by the original EPC. Additional databases may be searched on a case-by-case basis (e.g., if two or more of the specialty journals are not catalogued on Medline).

The option of the systematic review or update will be submitted to Contracts by the TOO after consultation with the EPC team.

Deliverable to AHRQ o Completed Topic Refinement Document, Parts 2 and 3 and alternative text (508 compliant) for Analytic Framework. Upload to the Secure Site and notify the TOO.

o Completed Surveillance Report. Preliminary sample templates on Secure site at:

Resources /EPC Process Resources /01. EPC Program Policies and Procedures /Surveillance for Update o

Resource o Topic Refinement Content Guidance Document on Secure Site at: Resources /EPC Process Resources /05. Topic Refinement /Topic Refinement Templates o Surveillance Update Guide with Attachments on Secure Site at: Resources /EPC Process Resources /01. EPC Program Policies and Procedures /Surveillance for Update https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=6907 https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=13632 https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=0 https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=6906 https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=6906 https://www.epc-src.org/src/secureEHC/content.cfm?AREA=1&FLDR=6907

11. Scientific Information Packets (SIP)

The SRC SIP Coordinator requests information from industry stakeholders on behalf of the EPCs. Centralizing the process in the SRC ensures that all requests are handled in the same manner and prevents industry from directly contacting the EPC working on the project.

The SIP coordinator will use the preliminary SIP list in the Topic Refinement Document. They will begin the necessary background work to prepare relevant SIP requests. SIP requests will be sent when the final protocol is posted on the EHC web site.

Resource o Topic Refinement Content Guidance Document on Secure Site at: Resources /EPC Process Resources /05. Topic Refinement /Topic Refinement Templates o “SIP_EPC Instructions” on the Secure Site at: Resources /EPC Process Resources /06. Scientific Information Packets (SIPs)

12. Public Posting, Topic Refinement Document Part 3 (KQ Posting Document)

The Key Question Posting Document will be posted to the EHC web site for solicitation of public comments for three weeks. For topics developed under the Technology Assessment Program the public comment period is 2 weeks.

The Surveillance Report will be posted on the EHC web site for informational purposes.

The EPC should e-mail (and include a link to the EHC website for the Key Questions) the Key Informants, nominator, and partners to thank them for their participation on the project and to offer them an opportunity to comment on the topic scope via the public review process.

If the EPC is contacted by any organization or industry representatives wanting to discuss the Key Question Posting Document, the EPC should refer the individual to the TOO and request any comments to be provided in writing via the public comment mechanism on the EHC web site.

13. Disposition of Topic Refinement Public Comments For TA Program

a. The EPC shall review public comments and provide a disposition of comments document to be sent to the TOO and CMS project staff. This disposition is an internal document, and will not be publically posted.

o Deliverable: Disposition of Topic Refinement Public Comments sent to

AHRQ TOO and associated CMS project staff and placed on the Secure Site.

b. The EPC shall host a Key Question Comments Call with the TOO and CMS project staff to discuss public comments, disposition of comments, suggest edits to the key questions or PICOTs and flag any preliminary issues for the protocol.

o Deliverable: Call minutes sent to call participants and placed on the Secure Site. Notify TOO.

o Meeting summary template on Secure Site at: Resources /EPC Process Resources /05. Topic Refinement /Call Resources

c. The EPC shall provide a revised Topic Refinement Part 3 Document with revised KQs, PICOTs based on disposition of comments, including a brief 1-2 paragraph summary of comments that resulted in revisions. This revised version will be posted on the TA program web site for informational purposes.

o Revised Topic Refinement Part 3 Document sent to AHRQ TOO and associated CMS project staff and placed on the Secure Site.

14. Development of Draft Protocol for Systematic Review Update Based on the highest priority Key Questions as discussed with the TOO, the EPC shall develop a draft update protocol for updating the systematic review using the Protocol Content Guidance.

The draft update protocol shall include a summary of the Surveillance report as a rationale for the update. The draft Update protocol should indicate that the report under development is an update of an existing report and provide a reference to the prior report. It should clearly indicate whether the key questions to be answered by the Update report are the same or if they vary how they vary and why. If the population or interventions or outcomes are different from the original report, this should also be explained and reasons for the changes/additions or deletions indicated.

Deliverable to AHRQ o Draft Update Protocol. Upload to the Secure Site and notify TOO

Resource

• Protocol Content Guidance on Secure Site at: Resources /EPC Process Resources /08.

Systematic Review Protocol

• Methods Guide and Medical Test Methods Guide

(http://effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?productid=318&pageaction=displayproduct and http://www.effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productid=558

Systematic Reviews and Updates

A systematic review examines available research literature thoroughly to assess and synthesize the evidence available to answer Key Questions that clinicians, patients, payers, policymakers, and other stakeholders have identified as important to their ability to make decisions about the topic of the review.

Items specified for delivery are reviewed by the Task Order Officer (TOO) before final acceptance. The EPC shall be required to make revisions as deemed necessary by the TOO to ensure a report of high quality. The EPC shall submit deliverables to the Secure Site, in the format and within the timeframes indicated. Due dates which fall on a weekend or holiday are due on the next business day.

Resource

• Documents describing the Systematic Review (SR) and Update process (including content guidance and examples) can be found on the EHC Secure Site at: Resources /EPC Process Resources

The Systematic Review Process and Deliverables:

1. Research Review Staffing Plan and Project Management file

The Staffing plan identifies key staff working on the project, and the Project Management file lays out the timeline for the project and specific deliverables.

Deliverable to AHRQ

• Completed Staffing Plan. This should list all project personnel, their roles, and the primary point(s) of contact for the project. Upload to the Secure Site and notify TOO

• Project management file with each deliverable and a deliverable date. Upload to the Secure Site and notify TOO

Resource o Staffing plan template on Secure Site at: Resources /EPC Process Resources /01. EPC Program Policies and Procedures /Work Files and Contract Process Forms

2. Staff Disclosure Forms, and Acknowledgement and Confidentiality Agreements

The EPC shall upload a signed Disclosure form and a signed Confidentiality agreement for all participants in the review. EPCs may choose to use web-based disclosure forms and confidentiality agreements. Staff members who have already submitted Disclosure forms for the Topic Development or Topic Refinement phase do not need to resubmit disclosures unless they have acquired new interests or more than a year has elapsed since the previous disclosure.

During the course of the project, an updated form should be submitted if a new conflict arises.

The TOO will review and approve the participation of all proposed staff before their participation is initiated.

The EPC Director is responsible for assembling the team who will work on Topic refinement and SR or Technical briefs. The Director should review the Disclosure forms for potential financial or non-financial conflicts of interest. For any potential conflicts of interest, the EPC Director may prepare a document describing the potential conflict and any plans in place to manage the potential conflict of interest. The EPC will submit this document to the TOO for discussion with EPC program director. The TOO will request a call with the EPC director if further clarification is needed or if the TOO has concerns about the participation of an individual with disclosed conflicts of interest.

The EPC should receive acknowledgment of the receipt and acceptance of the Disclosure forms, and if relevant the plan for management of any conflicts of interest from the TOO. If the EPC does not receive this within a week, follow-up should ensue on the part of the EPC.

The Eisenberg Center will send Disclosure forms directly to the TOO prior to the kick-off call.

The EPC shall maintain confidentiality on the findings of the systematic review. The EPC is not to present or publish findings until the report is released for publication by AHRQ except with explicit approval by the TOO.

The EPC program has specific venues for solicited expert input through engagement of Key Informants, Technical experts, and peer reviewers, as well as solicited public comment on the key questions and draft report. The EPC shall refer any unsolicited inquiries or comments outside of these structured processes to the TOO.

Deliverable to AHRQ o Signed Disclosure forms. Upload to the Secure Site and notify TOO o Signed Confidentiality Agreement. Upload to the Secure Site and notify TOO

Resource o AHRQ Evidence-based Practice Center Policy on Financial and Non-financial Interests, and Disclosure from for COI on Secure Site at: Resources /EPC Process Resources /01. EPC Program Policies and Procedures o EPC Review Team Confidentiality Agreement on Secure Site at: Resources /EPC

Process Resources /01. EPC Program Policies and Procedures o Web-based Disclosure Form and Confidentiality Agreement. Contact the SRC for information and access.

3. Technical Expert Panel (TEP)

If this has not already been done during Topic Refinement, the EPC will identify individuals to serve on a TEP. The TOO and partner may also suggest individuals to serve on the TEP. The EPC may choose to modify the proposed TEP list based on input during topic refinement. The TEP informs the scientific content of the research review and provides input to the EPCs throughout the evidence review process, particularly relevant to the protocol for conducting the review. Note: For TA reports, the TEP is comprised solely of CMS staff.