EPC IV Attachment 7.pdf

EPC Procedures Guide For Effective Healthcare Program

Systematic Reviews

Center for Outcomes and Evidence

Agency for Healthcare Research and Quality

EPC Procedure Guide

Please refer to the Secure Site under Resources/EPC Process Resources to identify current deliverable templates

Version 10 – 10/20/2011

Version History

Date Version Changes

10/20/11 10 1) Shorten timeline for Topic Refinement

2) Definition of process for managing conflicts of interest

3) Definition of process for responding to AE and TOO comments while maintaining EPC independence.

4) Clarify role of TEP and KI

5) Clarify post report translation and dissemination work.

6) Clarify lines of communication at kick-off call

9/26/11, 10/20/11, 9 1) EPC inputs peer reviewers into scholar one

2) EPC refers any outside contacts to TOO

3) De-identified Topic briefs are posted publicly

4) EPC may submit journal manuscript at draft stage, updated instructions on coordination of publication

5) TBD process for management of conflicts

6) Modify language related to relay of peer and public comments on draft reports to EPC

TBD process for clarifying disputes between EPC and AE or TOO.

Table of Contents

TOPIC DEVELOPMENT

Overview:

Resources:

The Topic Development Process and Deliverables:

1.Project Management Work Plan

2. Staff Disclosure

3. Targeted Topic Development Discussions with Specific Stakeholders or Experts

4. Development of the Topic Brief

5. Presentation to Topic Triage

6. Consultation on Nomination Summary Document

7. Responding to feedback generated by Nomination Summary Document

Submission of Topic Brief draft to TOO

TOPIC REFINEMENT

Overview:

Resources:

The Topic Refinement Process and Deliverables:

1.Topic Refinement Project Management Work Plan

2.Staff Disclosure

3.Kick-Off Call and Summary

4. Develop a list of Key Informants (KI) and Obtain Disclosure of Interest Forms

5. Literature Review and Topic Refinement Brief, Part 1

6. Host Key Informant Conference Call(s)

7. Final Topic Refinement Brief and Key Question document

COMPARATIVE EFFECTIVENESS REVIEWS AND UPDATES

Overview:

Resources:

The CER Process and Deliverables:

1. Research Review Project Management Work Plan

2. Staff Disclosure

3. Key Question Comments Discussion Call, Kick Off Call, and Key Question Disposition of Comments

4. Draft Protocol (follow-on from Topic Refinement Brief)

5. Proposed Technical Expert (TEP) Panel list

6. TEP Disclosure of Interest

7. First TEP Call (required)

8. Final Protocol for Posting

9. Literature Review

10. Second and Additional TEP Calls (optional)

11. Monthly Conference Calls and Status Reports

12. Peer Review

13. Draft Comparative Effectiveness Review for Peer Review

14. Posting for Public Comment

15. Update of Literature

16. Disposition of Peer Reviewer and Public Comments

17. Final Comparative Effectiveness Review

18. Journal Article

19. Assistance with Translation

20. Self-Evaluation Form

21. Presentations on the CER and submission of abstracts

TECHNICAL BRIEFS

Overview:

Resources:

1. Project Management Work Plan and Staff Disclosures

2. Kick-off Call

3. Preliminary Literature Review, Development of Protocol and Key Informant list

4. Research Phase: Systematic Literature Scan & Key Informant Interviews

5. Monthly Conference Calls and Status Reports

6. Peer Review

7. Draft Technical Brief for Peer Review and Posting for Public Comment

8. Update of Literature

9. Disposition of Peer Reviewer and Public Comments

10. Final Technical Brief

11. Journal Article

12. Assistance with Translation

13. Self-Evaluation Form

14. Presentation Slides and Submission of Abstracts

Topic Development Overview:

The goal of Topic Development and Topic Triage is to develop and select topics for research. Topic Development involves further describing a topic nomination to identify the clinical and/or policy context surrounding the nomination, evaluate the topic’s fit with the EHC program topic selection criteria and synthesize this information into a Topic Brief that provides the necessary background information for subsequent discussion and voting by the Topic Triage Group. The EHC Program Topic selection criteria are: 1. Appropriateness, 2.

Importance, 3. Duplication, 4. Feasibility, and 5. Potential Impact. These developed topics are then presented to the Topic Triage Group who then recommends topics for further research. Topics selected to go forward for a research review are then further refined to produce focused key questions (described under Topic Refinement activities).

At the start of Topic Development, the EPC will receive a completed Topic Nomination Form and any supporting documents submitted by the nominator.

Items specified for delivery are subject to the review of the TOO before final acceptance. Deidentified Topic Briefs will be posted publicly on the EHC website. The EPC shall be required to make revisions as deemed necessary by the TOO to ensure a topic brief of high quality. The EPC shall submit items to the specified recipient, usually to the EHC Secure Site (www.kpchr.org/ehc) with e-mail notification to the TOO, in the format and within the timelines indicated. Due dates which fall on a weekend or holiday are due on the next business day.

Resources:

Documents describing the Topic Development process (including templates and examples) can be found on the EHC Secure Site:

https://www.kpchr.org/ehc/Resources/Topic Nomination Development Training Materials

The folder also contains the training on topic development (in Power Point) presented by the SRC at the November 2009 EPC Directors meeting.

The Topic Development Process and Deliverables:

1.Project Management Work Plan The purpose of the work plan is to lay out the timeline for the project and specific deliverables, as well as identify key staff working on the project.

Deliverable to AHRQ:

Project Management Work Plan in Microsoft Project or Excel for each deliverable and a deliverable date.

Suggested timelines are included in the description of each deliverable.

The Project Management Work Plan should list all personnel assigned to the project, their roles and identify the primary point(s) of contact for the project.

http://www.kpchr.org/mma� https://www.kpchr.org/ehc/Apps/documentation.aspx?ContentAreaID=2933&ParentContentAreaID=3&ContentAreaRootID=3� https://www.kpchr.org/ehc/Apps/documentation.aspx?ContentAreaID=2933&ParentContentAreaID=3&ContentAreaRootID=3� https://www.kpchr.org/ehc/Apps/documentation.aspx?ContentAreaID=2933&ParentContentAreaID=3&ContentAreaRootID=3�

Personnel can include a principal investigator who oversees the scientific content of topic development, a project director who is responsible for management of Topic Development, a research associate who conducts the research and helps prepare the topic briefs, a research librarian, and clinical experts who provide clinical feedback.

Timeline:

Within one week of topic assignment.

Resources:

Microsoft Project template for Topic Development Staffing plan template

2. Staff Disclosure The EPC shall send to the TOO a copy of a signed Disclosure of Interest forms for each individual who will assist in the Topic Development process including core staff. The TOO will review and approve the participation of all proposed staff before their participation is initiated.

Deliverable to AHRQ:

Signed Disclosure of interest forms to TOO

Timeline:

Within two weeks of topic assignment and prior to beginning work.

Resource EPC Program Disclosure Policy and Form

3. Targeted Topic Development Discussions with Specific Stakeholders or Experts The purpose of these discussions is to clarify the contextual issues driving the nomination.

Discussions are to be limited and efficient, focused solely on providing initial direction in developing the topic to be responsive to the decisional needs of stakeholders. Input can be garnered from local clinical experts (including primary care providers and sub specialists), the nominator, or other stakeholders readily available to the EPC. Since discussions are limited and for the purpose of background, Disclosure of Interest forms are not needed from these individuals. This input can be used to understand aspects of the topic such as practice variation, clinical uncertainty, appropriate comparators and important subpopulations.

Discussions with the nominator may be helpful for clarification, context, and to understand specific requirements (for example, if the nominator indicates that they are requesting a review to support guideline development, it is helpful to know when they hope to issue the guideline).

Deliverable to AHRQ:

A summary of the discussions should be documented on the topic brief cover sheet

(topic brief described below).

Timeline:

Discussions should occur during the Topic Development process and prior to final submission of the topic brief

Resource:

Sample topic briefs

4. Development of the Topic Brief

The three main parts of the Topic Brief are:

The Topic Brief is a packet of materials completed by the EPC for each nomination as a part of Topic Development and whose purpose is to inform the topic selection discussion.

De-identified Topic Briefs will also be posted publicly to the EHC website.

1.)

The purpose of the cover sheet is to summarize the nomination and the key considerations supporting the EPC staff recommendation as to whether a topic should go forward as a research review. The Cover Sheet should therefore include:

Topic Triage Cover Sheet

• Documentation of targeted discussions with specific stakeholders or local topic experts held to clarify the topic nomination and provide direction on the refinement of the topic.

• An outline of key considerations and discussion points for the Topic Triage selection discussion.

• A Draft/Preliminary PICO(TS)

• If applicable, a list of 2-3 relevant publications to be reviewed by voting members in order to increase familiarity with the topic (i.e. review articles, not important guidelines or studies unless they are the best way to educate the Topic Triage members on the topic);

• A final staff recommendation, by the EPC, including the disposition code (see reference materials for code sheet)

2.)

This purpose of this table is used to document a nomination’s fit with each of the program’s selection criteria.

Selection Criteria Table

These criteria are: 1. Appropriateness, 2. Importance, 3. Duplication, 4. Feasibility, and 5.

Potential Impact. Key points from this document can be summarized in the topic triage coversheet.

3.) Existing Guidance Table

The purpose of this table is to document existing evidence on the topic including reviews, guidelines, policies, and primary research.

Three-part Topic Brief.

During the EPC’s first experiences with Topic Development, Topic Briefs will be reviewed by the Scientific Resource Center (SRC) as well as the TOO and then scheduled by the SRC/AHRQ for presentation at a Topic Triage meeting. Once approved by the TOO, the SRC will post the documents on the EHC secure site in the appropriate Topic Triage meeting folder and will send an email to the Topic Triage group alerting them to the availability of the documents on the EHC secure site.

Timeline:

Within 6 weeks of topic assignment

Resources:

Templates for each of the three documents Examples of completed documents Guidance for assigning Topic Triage Coding Recommendations

5. Presentation to Topic Triage The purpose of the Topic Triage Presentation is to summarize the Topic Brief document and answer questions about the scope of the nomination.

All of the topics developed by the EPC will be presented to a Topic Triage Group by the

EPC. The EPC may assume that the Topic Triage members have reviewed the Topic Brief in advance of the meeting. The Topic Triage Group, comprised of members from AHRQ staff, the SRC, and the Eisenberg Center (EC) meets at least once per month (typically the first Wednesday of the month between 12-2 pm EST) to review, discuss, and vote whether a given topic nomination, informed by the Topic Brief, will go forward as a research review for the next step of Topic Refinement.

Deliverable to AHRQ:

A 2-3 minute presentation during topic triage meeting. The EPC should also be available to answer any questions during the group discussion.

Timeline:

Presentation should take place within one to two months of submission of brief

(scheduling of the presentations will be coordinated by AHRQ and the SRC).

Topic triage meetings generally occur on the first Wednesday of the month between 12-

2 pm EST. Meetings occur via a conference call set up by the SRC.

Resource:

Participate as listener in Topic Triage meeting. Contact your TOO or epc@ahrq.gov for dial-in information.

6. Consultation on Nomination Summary Document The purpose of the Nomination Summary Document is for transparency; to inform the public of the outcome of each topic nomination through a posting on the EHC public web site.

After the Topic Triage meeting, the SRC will prepare minutes of the Topic Triage meetings and finalize the Topic Triage Coversheet. The SRC will also create a Nomination Summary Document that outlines the disposition for each triaged nomination. The EPC will be expected to review the Nomination Summary Document(s) prior to posting. Finalized cover sheets (with the disposition and meeting minutes incorporated) will be posted by the SRC on the EHC Secure Site. The Nomination Summary Document describing the disposition will be posted by AHRQ/EC on the EHC website and will be sent to the nominator by the SRC.

Deliverable/Activity:

Review a Nomination Summary Document at the request of the SRC.

Timeline:

Will be scheduled by SRC, usually within 2 weeks of Topic Triage meeting.

Resource:

Sample Nomination Summary Documents.

7. Responding to feedback generated by Nomination Summary Document The purpose of responding to feedback generated by Nomination Summary Documents is to ensure that the EPC program is responsive to the public and that all decisions are transparent.

If a nominator or member of the public sends a comment or question concerning a

Nomination Summary Document, the SRC may ask the EPC to draft a response based on their greater knowledge of the topic. The response will then be reviewed by the SRC and the TOO before the SRC sends the response to the person who submitted the comment/question.

Deliverable/Activity:

Draft response at the request of the SRC.

Timeline:

Will be requested by SRC, usually sent within 1 week of receipt of question.

mailto:epc@ahrq.gov�

Feedback and Response –Nomination Summary Documents.

Topic Development Sample Schedule of Deliverables

Date of Assignment: 1/1/2011

Task Format Work Interval from Award Date

Date Due

Pre-Kickoff Call Submit Microsoft Project timeline and staffing plan

Electronic 1 week 1/8/2011

Submit Staff DOI forms Electronic 2 weeks 1/14/2011 Kick-off Call Hold call with TOO and EC to discuss goals/objectives of project, key questions, etc.

2 weeks 1/15/2011

Submit summary of call's discussion decisions, to call participants 1 week from kick-off call.

Electronic 1 weeks after kick-off call

1/22/2011

Topic Brief Targeted Topic Development Discussions with Specific Stakeholders or Experts

Prior to submission of draft

Submission of Topic Brief draft to TOO

6 weeks 2/15/2011

Submit final draft to SRC 7 weeks 2/22/2011 Presentation to Topic Triage 8 or more weeks 3/2/2011

Consultation on Nomination Summary Document

2 weeks after Topic Triage

3/16/2011

Responding to feedback generated by Nomination Summary Document variable

Topic Refinement

The purpose of Topic Refinement is to finalize the scope and to tighten up the key questions and PICOTS for Comparative Effectiveness Reviews so they are informative and reflect stakeholder decisional dilemmas, and are ready for public comment. Technical Briefs do not undergo Topic Refinement.

Topic Refinement work will be assigned shortly after Topic Triage decisions are finalized

(usually 2 weeks). Assignments may be direct or competed and decisions are based on available expertise and capacity. The timeline goal for Topic Refinement is 4 months.

The key deliverable for Topic Refinement has three parts:

1.) Topic Refinement Brief, Part 1 Initial topic refinement moves the topic from the nomination stage to a point where it is ready to be discussed with key informants. The background and clinical context is expanded preliminary key questions are clarified, and key considerations and discussion questions for subsequent Key Informant discussions are described. An important aspect to the initial topic refinement is the proposal of an analytic framework.

2.) Topic Refinement Brief, Part 2 The last section of the Brief summarizes the discussions with key informants and creates a list of known pharmaceutical or device companies or other professional entities or researchers from whom Scientific Information Packets (SIP) should be requested once the key questions are finalized (i.e. when the protocol is finalized and posted)

3.) Key Question Posting Document A Posting Document for public comment on key questions is submitted as a separate file using a template document.

Items specified for delivery are subject to the review of the TOO before final acceptance. The EPC shall be required to make revisions as deemed necessary by the TOO to ensure a report of high quality. The EPC shall submit items to the specified recipient, usually to the EHC Secure Site (www.kpchr.org/ehc) with e-mail notification to the TOO, in the format and within the timelines indicated. Due dates which fall on a weekend or holiday are due on the next business day.

Resources:

Documents describing the Topic Refinement process (including templates and examples) can be found on the EHC Secure Site: https://www.kpchr.org/ehc/Resources/EPC Process Resources/2. Topic Refinement http://www.kpchr.org/mma� https://www.kpchr.org/ehc/Resources/EPC%20Process%20Resources/2.%20Topic%20Refinement� https://www.kpchr.org/ehc/Resources/EPC%20Process%20Resources/2.%20Topic%20Refinement�

The Topic Refinement Process and Deliverables:

1. Topic Refinement Project Management Work Plan The purpose of the topic refinement project management work plan is to lay out the timeline for the project and specific deliverables, as well as identify key staff working on the project.

Deliverable to AHRQ;

Project Management Work Plan in Microsoft Project should list all personnel assigned to the project, their roles and identify the primary point(s) of contact for the project.

Timeline:

Within one week of topic assignment for refinement.

Resources:

Microsoft Project template for Topic Refinement Staffing plan template

2. Staff Disclosure The EPC shall send to the TOO a copy of a signed Disclosure of Interest forms for each individual who will assist in the Topic Refinement and CER development process (this includes participants from the Eisenberg Center and core EPC staff). Staff who already submitted Disclosure of Interest forms for the Topic Development phase do not need to resubmit disclosures unless they have acquired new interests or more than a year has elapsed since the previous disclosure. The TOO will review and approve the participation of all proposed staff before their participation is initiated. The EPC Director is responsible for assembling the team who will work on Topic refinement and CER or Technical briefs. The Director should review the Disclosure of Interest forms for potential financial or non-financial conflicts of interest. For any potential conflicts of interest, the EPC Director may prepare a document describing the potential conflict and any plans in place to manage the potential conflict of interest. The EPC will submit this document to the TOO for discussion with EPC program director and COI team. The TOO will request a call with the EPC director if further clarification is needed or if there are concerns for the participation of a key informant with disclosed conflicts of interest.

The EPC should receive acknowledgment of the receipt and acceptance of the COIs, and if relevant the plan for management of any conflicts of interest from the TOO. If the EPC does not receive this within a week, follow-up should ensue on the part of the EPC.

The Eisenberg Center will send Disclosure of Interest forms directly to the TOO prior to the kick-off call.

Submission of signed Disclosure of Interest forms

Timeline:

Within two weeks of topic assignment for refinement and prior to beginning work.

Resource:

EPC Program Disclosure Policy and Form

3. Kick-Off Call and Summary The EPC shall convene and lead a conference call with the TOO and other EPCs with relevant areas of concentration and/or expertise if appropriate. The Eisenberg Center (EC) is invited in a listener role. Optional participants include appropriate AHRQ staff or other individuals involved in Topic Development. The purpose of the call is to:

• Review specific elements of work to be performed

• Identify potential stakeholders to serve as Key Informants during the Topic

Discussion Conference call

• Describe a beginning strategy for working with Key Informants

• Establish lines of communication between the EPC, TOO, stakeholders, other AHRQ staff, and partners, including professional organizations and the USPSTF or other federal partners. The TOO should be copied in all email communications between the EPC and partner organizations. If there are rare exceptions when the EPC has a call with the partner without the TOO, the EPC should send a written summary to the

TOO.

Deliverable to AHRQ:

The EPC shall provide an electronic summary of the discussion to everyone who participated in the call, as well as posting it on the Secure Site. The EPC may choose to provide this summary to Key Informants before the Key Informant call. (see below).

Timeline:

Kick-Off Call: within two weeks of topic assignment for refinement Call Summary: within one week of call

Resources:

Topic Refinement Content Guidance Guide for engaging stakeholders - materials under development by SRC (Available fall

2010 on secure site)

4. Develop a list of Key Informants (KI) and Obtain Disclosure of Interest Forms In consultation with the TOO, the EPC shall identify and suggest a small number of individuals (6-8) including patients and caregivers, practicing clinicians, relevant professional and consumer organizations, purchasers of health care, and others with experience in making health care decisions relevant to the topic. The primary role of the KI is to identify the particular questions to be reviewed. The EPC is seeking input from KIs on the particular populations, interventions, comparisons, or outcomes of particular interest to them. Specifically, the KIs may provide input on:

• Outcomes of particular importance to be graded

• Comparisons of particular interest or uncertainty which users must decide between

• Populations which should be considered distinct from one another when comparing interventions or tests. Identify specific populations in which there is particular controversy or uncertainty

Thus, KIs are distinct from the Technical Expert Panel (TEP) which is constituted to inform the scientific processes of the subsequent research review. The input from the KIs will assist the EPC in refining Key Questions and the Analytic Framework. Depending on their expertise, some members of the KI group may go on to serve as members of the TEP.

Disclosure of Interest forms should be obtained for KIs. CVs should also be obtained from the KIs with the exception of patients or similar participants who may not have readily available CVs. The requirement of disclosure in this instance is made in the interest of transparency and to help create a balance of perspectives among the Key Informants. KIs should be advised that their names and contact information will be shared with the SRC for potential consultation on future work unless they opt out. In addition, obtain names and completed Disclosure of Interest forms from any participants from the EC and EPC who may not have taken part in previous calls and submit to the TOO for approval. The PI of the project is responsible for assembling the lists of potential KI and TEPs. The PI should review the Disclosure of Interest forms for potential financial or non-financial conflicts of interest. While, in general, Key Informants may have conflicts of interest, if there are any declared interests of particular concern,, the PI may prepare a document (in consultation with the EPC director) describing the potential conflict and any plans in place to manage the potential conflict of interest. The EPC will submit this document to the TOO for discussion with EPC program director and COI team. The TOO will request a call with the EPC director if further clarification is needed or if there are concerns for the participation of a key informant with disclosed conflicts of interest.

The EPC should receive acknowledgment of the receipt and acceptance of the COIs, and if relevant the plan for management of any conflicts of interest from the TOO. If the EPC does not receive this within a week, follow-up should ensue on the part of the EPC.

NOTE: Individuals may not serve as KIs until their signed Disclosure of Interest forms have been submitted to AHRQ and are approved.

The EPC shall submit the names and completed Disclosure of Interest forms and CVs if applicable for proposed KIs to the TOO for review and approval.

Timeline:

Prior to Topic Discussion call

Resource:

EPC Program Disclosure Policy and Form Key Informant resources on the Secure Site

5. Literature Review and Topic Refinement Brief, Part 1 In order to have a useful and effective Key Informant call, everyone involved needs to be familiar with the basic facts about the topic and proposed research. Therefore, the purpose of Part 1 of the Topic Refinement Brief is to lay the groundwork for a productive discussion with the Key Informants. The Topic Refinement Brief Content Guidance describes the essential elements of this document: information about the original nomination, a summary of the nomination, a description of background and clinical context, preliminary key questions, a draft analytic framework, and key considerations for KIs. This portion of the document should be used to initiate and not limit discussions with the Key Informants.

In order to produce Part 1 of the Topic Refinement Brief, the EPC will systematically search and review the scientific evidence in a targeted manner, to evaluate the appropriateness and scope of a review on this topic area, to identify new or ongoing research in this area, and to assess the variance, if any, of the evidence according to age, gender, and race/ethnicity. This literature review is not intended to be synthesized, but to inform the scope of the topic.

Deliverable to AHRQ:

Part 1 of the Topic Refinement Brief. This deliverable should be submitted to the Secure

Site www.kpchr.org.ehc and the TOO notified.

Timeline:

2 months from start date and prior to KI topic discussion call(s).

Resource:

Topic Refinement Content Guidance

6. Host Key Informant Conference Call(s) The EPC shall invite participants, schedule, and lead conference calls to discuss the appropriateness and scope of the Key Questions and PICO. Participants in the call include the TOO, the topic nominator, the selected KIs, and EC staff.

Logistically, several calls may be needed to both meet the scheduling needs of KIs and to http://www.kpchr.org.mma/� adequately address the topic under refinement. If multiple calls are necessary, the EPC may consider holding the patient or consumer call separately, since patients or consumers may find it difficult to express their views during a call dominated by other seeming experts.

Deliverable to AHRQ:

A summary of the call(s) will be documented in the Topic Refinement Brief deliverable.

Submit summary of call discussions to call participants and submit a copy to secure site.

Timeline:

After Part 1 of the Topic Refinement Brief has been approved by TOO.

Resource:

Stakeholder Engagement Modules on Secure Site Topic Refinement Process Guide on Secure Site Key Informant and Key Informant Call resources on Secure Site

7. Final Topic Refinement Brief and Key Question document The Final Topic Refinement Brief includes Part 2, which summarizes the discussions with KIs, creates a list of known pharmaceutical or device companies or other professional entities or researchers from whom Scientific Information Packets (SIP) should be requested.

and revises aspects of Part 1 that should be amended or supplemented. The Topic Refinement Brief may include references and other relevant attachments. The Key Question Posting Document and the alternative text for the Analytic Framework will undergo internal review CERs and Updates will be posted on www.effectivehealthcare.ahrq.gov for public comment for a period of 4 weeks. The EPC should forward the link for the KQ Posting Document to KI and the nominator (if appropriate) to give an opportunity for public comment.

Deliverable to AHRQ:

Completed Topic Refinement Brief, Key Question Posting Document and alternative text (508 compliant) for Analytic Framework. This deliverable should be placed on the Secure Site and the TOO and EPC Coordinator (EPC@ahrq.gov) should be notified

Timeline:

4 months from assignment for refinement

Resources:

Topic Refinement Content Guidance Key Question Posting Document Content Guidance http://www.effectivehealthcare.ahrq.gov/� mailto:EPC@ahrq.gov�

UPDATE REPORTS: In order for the Update reports to accurately reflect current knowledge, the report may be expanded during the Topic Refinement period to include additional key questions not covered in the original report. The population, interventions and health outcomes covered in the report may also be expanded to reflect an increase in available knowledge and treatments. Additional analyses or new systematic review methods may also be included that reflect evolving methods work in development of CERs. Provide details on any changes in the key questions from the original report. The document should indicate that this is an update of an existing report and provide a reference and date of the prior report. It should clearly indicate whether the key questions to be answered by the Update report are the same or if they vary, how they vary and why. If the population or interventions or outcomes are different from the original report, this should also be explained and reasons for the changes/additions or deletions indicated.

Topic Refinement

Task Format Work Interval from Award Date

Date Due

Pre-Kickoff Call Submit Microsoft Project timeline and staffing plan

Electronic 1 week 1/8/2011

Submit Staff DOI forms Electronic 13 days 1/14/2011 Kick-off Call Hold call with TOO and EC to discuss goals/objectives of project, key questions, etc.

2 weeks 1/15/2011

Submit summary of call's discussion decisions, to call participants 1 week from kick-off call.

Electronic 1 weeks after kick-off call

1/22/2011

Key Informants (KIs) Submit names and CVs of proposed Key Informants to TOO.

Electronic prior to Topic Discussion call

Send signed Disclosure of Interest forms for each KI to the TOO.

Paper or electronic prior to Topic Discussion call

Topic Refinement Brief Part One Reference list with a 2-3 paragraph summary of the literature search results and discussion of the issues considered.

Electronic 6 weeks 2/15/2011

Topic Discussion Call(s) Hold Topic Discussion call(s) 2 months 3/1/2011 Submit summary of call discussion and decisions to call participants, including KIs who could not participate, and TOO.

Electronic 1 weeks after call

3/8/2011

Topic and Key Question Development Report

Complete and submit the final Posting form to TOO and EPC Coordinator (EPC@ahrq.gov).

Electronic 3 months 4/1/2011

Comparative Effectiveness Reviews and Updates

A comparative effectiveness review examines available efficacy data thoroughly to assess and synthesize the evidence available to answer Key Questions that clinicians, patients, payers, policymakers, and other stakeholders have identified as important to their ability to make decisions about that medical intervention. After undergoing refinement and posting for public comment, topics will continue for systematic review once comments and EPC program capacity are considered by AHRQ. Assumptions are that the same EPC that refined the topic will conduct the review unless capacity issues preclude that. The timeframe for that consideration is usually about two weeks. Topics for review will be assigned by the

TOO.

Items specified for delivery are subject to the review of the TOO before final acceptance. The EPC shall be required to make revisions as deemed necessary by the TOO to ensure a report of high quality. The EPC shall submit items to the specified recipient at the e-mail address specified earlier, in the format and within the timeframes indicated.

Due dates which fall on a weekend or holiday are due on the next business day.

Resources:

Documents describing the CER and Update process (including content guidance and examples) can be found on the EHC Secure Site: https://www.kpchr.org/ehc/Resources/EPC Process Resources

The CER Process and Deliverables:

1. Research Review Project Management Work Plan The purpose of the research review project management work plan is to lay out the timeline for the project and specific deliverables, as well as identify key staff working on the project.

Deliverable to AHRQ:

Project Management Work Plan in Microsoft Project for each deliverable and a deliverable date.

Suggested timelines are included in the description of each deliverable. The Project

Management Work Plan should list all personnel assigned to the project, their roles and identify the primary point(s) of contact for the project.

Timeline Within one week of topic assignment for the research review.

https://www.kpchr.org/ehc/Resources/EPC%20Process%20Resources� https://www.kpchr.org/ehc/Resources/EPC%20Process%20Resources�

Microsoft Project template for CERs or Updates Staffing plan template

2. Staff Disclosure The EPC shall send to the TOO a copy of a signed Disclosure of Interest forms for all participants in the review. Staff who already submitted Disclosure of Interest forms for the Topic Development or Topic Refinement phase do not need to resubmit disclosures unless they have acquired new interests or more than a year has elapsed since the previous disclosure. The TOO will review and approve the participation of all proposed staff before their participation is initiated. The EPC Director is responsible for assembling the team who will work on Topic refinement and CER or Technical briefs. The Director should review the Disclosure of Interest forms for potential financial or non-financial conflicts of interest. For any potential conflicts of interest, the EPC Director may prepare a document describing the potential conflict and any plans in place to manage the potential conflict of interest. The EPC will submit this document to the TOO for discussion with EPC program director and COI team. The TOO will request a call with the EPC director if further clarification is needed or if there are concerns for the participation of a key informant with disclosed conflicts of interest.

The EPC should receive acknowledgment of the receipt and acceptance of the COIs, and if relevant the plan for management of any conflicts of interest from the TOO. If the EPC does not receive this within a week, follow-up should ensue on the part of the EPC.

The Eisenberg Center will send Disclosure of Interest forms directly to the TOO prior to the kick-off call.

Deliverable to AHRQ:

Signed Disclosure of Interest forms to TOO

Timeline:

Within two weeks of topic assignment for research review and prior to beginning work.

EPC Program Disclosure Policy and Form

3. Key Question Comments Discussion Call, Kick Off Call, and Key Question Disposition of Comments

Once all of the Key Question public comments have been received, the EPC shall review the public comments and convene and lead a conference call with the Task Order Officer (TOO) and other EPCs as appropriate with relevant areas of concentration. The Eisenberg Center is invited to participate in a listening role. Optional participants include other appropriate AHRQ staff. The purpose of the call is to review public comments, discuss disposition of comments and suggested edits to the key questions or PICOTS.

The call can also serve as the CER kick off call and can be used to discuss suggested approaches for the protocol, though finalization of the protocol will be conducted via TEP input. Participants may also suggest Technical Expert Panel members and peer reviewers at this time. The call should also establish lines of communication between the EPC, TOO, stakeholders, other AHRQ staff, and partners, including professional organizations and the USPSTF. In general the TOO should be copied in all communications.

The EPC shall provide an electronic summary of the discussion to participants and place a copy of the summary on the secure site. Disclosure of Interest forms should be completed for each person contributing to the content of the report including EPC staff, consultants, subcontractors, and others.

Upon completion of the call, the EPC will describe and document the disposition of all Key Questions in a disposition of comments document. In some cases the dispositions can be finalized after input from the TEP.

Deliverables to AHRQ:

Meeting minutes from the call to be sent to call participants and placed on the Secure

Site and notify TOO.

Timeline:

Call and Disposition of Comments table: 6 weeks after Key questions are posted Call summary one week after call

Resources:

Kick-off call discussion points Key Question Posting Brief Content Guidance Disposition of Comments Table template

4. Draft Protocol (follow-on from Topic Refinement Brief) Develop and submit a draft protocol to the TOO. The protocol will include the following elements: a brief background of the topic (from the Topic Refinement Brief), description of the proposed literature search and review, including inclusion/exclusion criteria, criteria for evaluating the quality of studies, and rating the strength of the body of evidence. If the Key Questions have been significantly changed in response to KI or public input, it may be advisable to conduct an exploratory search to ensure that the new scope will not result in an unmanageable amount of literature, since any further changes will need to be discussed with the TEP.

Deliverable to AHRQ:

Draft Protocol

Timeline:

2 weeks after report kick-off call

Resource:

Protocol Content Guidance

5. Proposed Technical Expert (TEP) Panel list In consultation with the AHRQ TOO, identify individuals to serve on a TEP. The TEP is constituted to inform the scientific content of the research review and provide input to the EPCs throughout the evidence review process, particularly relevant to the protocol for conducting the review. Typically composed of 5 to 8 members, the Technical Expert Panels usually include one or more clinical providers, methods experts, and relevant professional society and health care purchaser representatives, and other content experts. The size and composition of the Technical Expert Panel are intended to create a balance between content and methodology expertise and the user’s perspective. During the course of developing evidence reports, Technical Expert Panels may assist the EPCs by providing input to:

• Focus the literature search by helping to identify search strategies and relevant grey literature.

• Identify inclusion/exclusion criteria.

• Clarify specific technical or clinical issues that may arise.

• Identify specific subgroups for which there may be heterogeneity of treatment effect.

The EPC would collect this information from studies to determine the applicability of evidence and perhaps to conduct sensitivity analysis or stratified analyses.

Deliverable to AHRQ:

Submit the list of proposed TEP members and their area of expertise and affiliations to the TOO for approval.

Timeline:

2 weeks after Report Kick-Off call

Resource none

6. TEP Disclosure of Interest When the TEP members are approved by the TOO, submit CVs, and signed Disclosure of

Interest forms for each expert to the TOO for final review and approval. If an individual has no business or professional interests to disclose, their signed Statement should include “none” in the appropriate places. The PI of the project is responsible for assembling the lists of potential KI and TEPs. The PI should review the Disclosure of Interest forms for potential financial or non-financial conflicts of interest. For any potential conflicts of interest, the PI may prepare a document (in consultation with the EPC director) describing the potential conflict and any plans in place to manage the potential conflict of interest. The EPC will submit this document to the TOO for discussion with EPC program director and COI team.

The TOO will request a call with the EPC director if further clarification is needed or if there are concerns for the participation of a key informant with disclosed conflicts of interest.

The EPC should receive acknowledgment of the receipt and acceptance of the COIs, and if relevant the plan for management of any conflicts of interest from the TOO. If the EPC does not receive this within a week, follow-up should ensue on the part of the EPC.

NOTE: Individuals may not serve as a TEP member (e.g., participate on conference calls, review interim draft products, etc.) until their signed Disclosure of Interest Form has been submitted to and approved by AHRQ.

Deliverable to AHRQ:

CVs and signed Disclosure of Interest Forms for each TEP member to be sent the TOO for final review and approval prior to the first TEP call.

Timeline:

Prior to first TEP call

Resource:

EPC Program Disclosure Policy and Form

7. First TEP Call (required)

The EPC will arrange an initial TEP conference call to discuss the draft protocol. In addition, the TEP will review the SIP list and any comments received regarding the SIP and revise if necessary. The Eisenberg Center is invited to all TEP calls.

The EPC should inform the TEP that they will be acknowledged by name in the report.

While they will not be specific peer reviewers, they can provide comments on the draft report through the public comment mechanism.

Meeting minutes distributed to all participants and a copy placed on the secure site.

Timeline:

2 months after topic assignment for research review.

Resource:

TEP call discussion points

8. Final Protocol for Posting The EPC shall revise their draft protocol to reflect discussions with the TEP. The Final

Protocol should contain a complete list of the drug or device interventions for which SIPs will be requested. Please be specific and provide both the generic and brand name of any drug under consideration. If additional drugs or devices are added to the review after this point, please notify Rose Relevo or Robin Paynter, SRC at: relevo@ohsu.edu or paynter@ohsu.edu. The SIPs requested by the SRC will be based on this list. It will take approximately 4 to 8 weeks for the SIP to be received from the time the Protocol is posted on the EHC website. The SRC will post the SIP material on the Secure Site when it is received.

The SRC uses the posting of the protocol as the trigger to initiate the SIP requests. If there are significant delays and the EPC would like the SRC to initiate the SIP request earlier, they should provide a complete list of interventions and contact the SIP coordinator.

UPDATE REPORTS: The Update Report protocol should indicate that the report under development is an update of an existing report and provide a reference to the prior report. It should clearly indicate whether the key questions to be answered by the Update report are the same or if they vary, how they vary and why. If the population or interventions or outcomes are different from the original report, this should also be explained and reasons for the changes/additions or deletions indicated.

Deliverable to AHRQ:

Submit the final protocol on the Secure Site and notify the TOO and EPC Coordinator

(EPC@ahrq.gov).

Submit Key Question comments disposition table on the Secure Site and notify the TOO and EPC Coordinator (EPC@ahrq.gov)

Due 2 weeks after TEP call

Resources:

Protocol Content Guidance SIPs listing template

9. Literature Review The EPC shall systematically search, abstract, review and analyze the scientific evidence mailto:relevo@ohsu.edu� mailto:paynter@ohsu.edu� for each question and the variance, if any, of the evidence according to age, gender, and race/ethnicity. The EPC shall review the EPC Methods Guide. Training modules are also available to EPC staff as well.

Resource:

EPC Methods Guide on EHC site

(http://www.effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productid=318)

EPC Training modules on EHC site (http://www.effectivehealthcare.ahrq.gov/index.cfm/slide-library-and-other-resources-for-faculty-researchers-and-policymakers/slide-library/#ahrq_training)

10. Second and Additional TEP Calls (optional) Additional TEP calls are often useful and held at the discretion of the EPC or as requested by the TOO. The second call typically occurs after reviewing the literature and learning what evidence is available to address the key questions and the TEP conference call agenda is to discuss any further clarification of the approach to data abstraction or presentation of data. TEP input should primarily with development of protocol, identifying literature sources, and methods of handling analyses. However the TEP should not review or be involved in the analyses of data.

Timeline:

Approximately 3 months after project start date for research review

Resource:

Second TEP call discussion points – Early Literature Check on Secure site

11. Monthly Conference Calls and Status Reports After completion of the first or second TEP call, the EPC shall participate in a monthly conference call with the TOO to summarize progress. The TOO may request that the agenda include attachments of working documents, such as reference lists, literature retrieval figures, draft summary tables, etc. Calls may be scheduled more or less frequently, as requested by the TOO. Within one week after the call, the EPC shall submit an electronic summary of the call to the TOO and place a copy on the secure site, which will serve as a status report to describe progress, any problems encountered and their resolution, any foreseeable problems, and any expected change to the delivery schedule.

Deliverable to AHRQ:

Minutes of call posted on Secure Site

Monthly

12. Peer Review The Peer Review Process will be a collaborative process where the EPC identifies potential peer reviewers and the Scientific Resource Center (SRC) manages the peer review process and delivers comments to the EPC at the conclusion of the review process. The SRC will manage the peer review invitation and collection of disclosure of conflicts process through ScholarOne Manuscripts.

About 3.5 – 4 months prior to the draft report due date, the SRC will request that the EPC upload their peer reviewer list and input the proposed names and contact information into ScholarOne.

To assure timely recruitment of at least five independent, unbiased reviewers, include the following information in the list:

• Identify 12 “first choice” reviewers

• Identify 5 alternate reviewers (invited only if first choice reviewers decline)

• In addition to the list of peer reviewers, include a list of the TEP members for the project

• Include name, title, email, affiliation and specialty of each proposed reviewer or

TEP member

• Label each name as first-choice, alternate or TEP

The TOO will review disclosures of conflicts to determine an appropriately balanced peer review panel. The TOO will be looking for 3-5 reviewers with appropriate mix of expertise (clinical, methodological, statistical) who are independent and unbiased on the review topic.

The TOO may add names of individuals to serve as peer reviewers, including individuals from other federal agencies and statistical/methods reviewers, before approving the final SRC invitation list. The TOO or the EPC may invite individuals who are not selected as peer reviewers to comment via the public website.

Deliverables

• Peer Reviewer List

• Submission of Reviewer Names and Contact Information in ScholarOne

The peer reviewer list is due at least 3 months before the draft report due date.

• EHC Secure Site: https://www.kpchr.org/ehc --Resources link on the sidebar menu

• Contact the SRC: EHCrev@ohsu.edu

13. Draft Comparative Effectiveness Review for Peer Review Prepare the draft evidence report and appendices. Draft reports are expected to be complete and of the same quality as a final report so that the peer review process can function effectively. The evidence report is to be concise and written in sufficient detail and clarity to be the primary document. It should include a synopsis of the scientific evidence on each key question and subsets, and reference to the linkages in the analytic framework, as well as grades for the strength of the evidence on each linkage. The discussion should be a synthesis of the results and include future research and conclusions sections. The draft CER includes an executive summary with specific content. Reports should follow AHRQ’s current required format, as posted on the secure site.

Submit the draft report through ScholarOne Manuscript Central and notify the TOO.

Draft CERs will be subject to concurrent peer review and public Web site posting for comments. Prior to public posting and before the document is sent out for peer review by the SRC, the document will be reviewed by the TOO and an Associate Editor. The AE and TOO are expected to reach a decision regarding revisions within 2 weeks of submission. The decision letter should distinguish between comments that are critical to be changed and those that are suggestions to for the EPC to consider with peer review. Critical changes may include inconsistencies within a report, non-adherence to accepted Methods Guidance, lack of clear description in methods, or inadequate analyses or address of scope of questions.

If there is significant disagreement on the revisions requested, the…