AHRQ-25-10002 Attachment 1A - EPC Procedures Guide.pdf

Table of Contents Introduction

A. Pre-Kickoff Call Activities

B. Post-Report Activities

C. Topic Development Brief

D. Topic Refinement and Systematic Review

E. Standalone Systematic Review (without Topic Refinement)

F. Living Review

G. Systematic Review Update

H. Rapid Review

I. Rapid Response

J. Technical Briefs, Scoping Reviews, and Evidence Maps

K. EPC Methods and Methods Guidance Reports

Appendix A: Topic Refinement Document

Introduction

Background

The AHRQ Evidence-based Practice Centers (EPCs) were created in 1997 to synthesize existing scientific literature about important health care topics and promote evidence-based practice and decision-making. Users of EPC reports include Federal agencies, guideline developers, clinicians, patients, policymakers, research funders, knowledge translators, and purchasers and payers of healthcare. EPC work has been the scientific foundation for development and implementation of clinical practice guidelines, review criteria, performance measures, other clinical quality improvements efforts, individual treatment decisions, coverage and other policies, and research agendas that lead to research funding opportunities.

All who work with the EPC Program have a shared goal: that the program’s products be useful, relevant, and understood by end-users. This Procedure Guide describes the evidence review development process for the EPC Program. This is intended to promote consistency and quality of AHRQ reports, and to foster a common understanding of the process.

This version of the Procedure Guide is intended to inform offerors for the EPC VII Request for

Proposal. While detailed, this document should not be considered comprehensive.

Roles and Responsibilities

AHRQ Evidence-based Practice Center (EPC) Division: Within AHRQ’s Center for Evidence and Practice Improvement (CEPI), the EPC Division manages most EPC contracts. A Task

Order Officer (TOO) is assigned to each review, provides some scientific oversight, and ensures that the review is consistent with EPC Methods and Procedures. The TOO also functions as the

Contracting Officer Representative (COR): an individual appointed by the Contracting Officer to assist in the technical monitoring or administration of a contract.

AHRQ Office of Communications (OC): OC develops and manages dissemination plans for EPC products; edits Final Reports for posting on the Effective Health Care (EHC) website and

National Library of Medicine (NLM) bookshelf; manages public affairs and media activities; and coordinates publication of the Final Report.

Evidence-based Practice Center (EPC): Under the EPC Program, AHRQ awards multi-year contracts to organizations. EPCs compete for task orders to develop evidence reports/products and participate in the development and piloting of methods and methods guidance. They also participate in the program’s quality assurance efforts to ensure consistency and promote quality of EPC reports.

Scientific Resource Center (SRC): This organization manages centralized activities for the EPC

Program. The SRC promotes the scientific credibility and independence of program products by managing peer and editorial review, providing technical support for topic nomination and selection processes, convening methods work, and supporting quality improvement efforts.

Introduction

Partner: AHRQ identifies a Partner prior to award of each project. The Partner may be a professional society, healthcare organization, or Federal agency that intends to use the EPC product. The Partner or its constituency is the primary audience for the report. The Partner will typically help identify the key decisional issues of the report. Partners are not part of the review team and are not considered authors of the evidence review.

Key Informants (KI) for systematic reviews: KI have perspectives that are interested in the decisional dilemmas of a report. They may help the EPC focus the review by identifying outcomes of interest, the types of comparisons of most concern, or the patient populations of greatest uncertainty. The KI panel may include patients, caregivers, practicing clinicians, researchers, professional or consumer organizations, purchasers of health care, and others who make health care decisions relevant to the topic.

Key Informants for technical briefs: KIs do not refine the guiding questions but rather inform them. KIs can contribute to an understanding of how the technology/intervention works, where it might fit into clinical care, and potential advantages or concerns. They may reveal questions and concerns that may not appear in the literature. KIs may include subject experts and end users of the technology/intervention, such as patients and caregivers, practicing clinicians, medical directors, relevant professional and consumer organizations, purchasers of healthcare, and others with experience in making healthcare decisions relevant to the topic.

Technical Expert Panel (TEP) for systematic reviews: The TEP informs the scientific approach for the systematic review outlined in the Protocol. The TEP usually includes clinical providers, methodologists, experts from relevant federal government agencies, representatives of relevant professional society and health care purchasers, and other content experts. The Technical Expert

Panel size and composition should create a balance between content and methodology expertise and the user’s perspective.

Peer Reviewers: Peer reviewers provide comments on a draft report. Peer reviewers have no professional or financial conflict with the content of the report as defined by the EPC Policy on

Financial and Nonfinancial Interests; and have not previously served as a KI or TEP for the draft report they review.

Useful Web Sites and Guides

AHRQ www.ahrq.gov

Effective Health Care Program www.effectivehealthcare.ahrq.gov

PROSPERO https://www.crd.york.ac.uk/prospero/

SRDR Plus https://srdrplus.ahrq.gov/

EPC Methods Guide for Effectiveness and

Comparative Effectiveness Reviews https://effectivehealthcare.ahrq.gov/products/co llections/cer-methods-guide

EPC Methods Guide for Medical Test Reviews https://effectivehealthcare.ahrq.gov/products/co llections/methods-guidance-tests http://www.ahrq.gov/ http://www.effectivehealthcare.ahrq.gov/ https://www.crd.york.ac.uk/prospero/ https://srdrplus.ahrq.gov/ https://effectivehealthcare.ahrq.gov/products/collections/cer-methods-guide https://effectivehealthcare.ahrq.gov/products/collections/cer-methods-guide https://effectivehealthcare.ahrq.gov/products/collections/methods-guidance-tests https://effectivehealthcare.ahrq.gov/products/collections/methods-guidance-tests

Pre and Post Report Activities

A. Pre-Kickoff Call Activities

1. Staffing Plan

Prior to the kick-off call, the EPC will submit a staffing plan, with the names and roles of EPC team members.

2. Deliverable Schedule

Within one week of the contract award date, the EPC must submit a deliverable schedule. This file lays out the timeline for the project and specific deliverables.

3. EPC Team Conflict of Interest (COI) Disclosure Forms and Confidentiality

Agreements

Each EPC project team member must submit a signed Conflict of Interest (COI) Disclosure form and the EPC Team Confidentiality Agreement. If potential conflicts are identified, the EPC must submit a mitigation plan. The TOO must review all forms and notify the EPC of approval (or objections) within one week after forms are received. Staff cannot participate in the project until approved. The EPC shall maintain confidentiality on the findings of the project. The EPC cannot present or publish any findings until a final product is publicly available.

4. Key Informant and Technical Expert Panel Lists (applies to systematic reviews, systematic review updates, and technical briefs)

i. Key Informants

The EPC shall submit a list of KIs to the TOO. The Partner and TOO may suggest additional or alternative individuals. Depending on their expertise, some KIs may also serve on the TEP.

Typically, the KI panel includes 1-2 individuals representing the Partner.

The KI panel size depends on the topic refinement size.

• Small topic refinement: 1-2 KIs who represent the Partner. A KI list is not required.

• Medium topic refinement: 3-6 KIs.

• Large topic refinement: 7-9 KIs.

ii. Technical Expert Panel

The EPC must submit a list of five to nine Technical Experts to the TOO. The Partner and

TOO may suggest additional individuals. Typically, the TEP includes 1-2 individuals representing the Partner.

iii. OMB Guidance, Paperwork Reduction Act

When organizing KI, TEP, or other calls, the EPC must abide by the Paperwork Reduction

Act (PRA), overseen by the Office of Management and Budget (OMB). The nine-person limit does not apply to U.S. federal government employees, or contractors within the EPC

Program.

Obtaining OMB approval to exceed the 9-person limit can be a lengthy process and is therefore not ideal, given the tight timeline for EPC Reports.

5. Public comment on Key Questions for Topic Refinement.

If a topic refinement is commissioned the key questions for the proposed review will be posted on the Effective Health Care website for three weeks of public comment prior to award of the task order. When the task order is awarded the public comments will be shared with the EPC.

B. Post-Report Activities

1. Systematic Review Data Repository Plus (SRDR+)

Final report data shall be entered into SRDR+ (https://srdrplus.ahrq.gov/) using data extraction forms or through upload of structured data. This is made freely available to the public upon report publication.

Investigators can upload data with Excel (.xlsx) or from Distiller SR. Upload of data in formats that cannot be accessed by SRDR+’s native search functions will be considered non-compliant

(such as .txt, .csv, .dta (Stata), .docx, and .pdf files). Instructions can be found on the SRDR+ website under the help tab.

The EPC shall email the TOO when the database has been uploaded and published in SRDR+.

The EPC shall add the TOO to the project on SRDR+ and assign them the role of Auditor.

2. Feedback Form

The EPC will complete a feedback form within one week after Final Report approval. Feedback is intended to identify issues encountered in the evidence review process. The EPC Program will use this to inform improvement in program guidance and the Methods Guide. It is not used for performance evaluation.

3. AHRQ Publication/Posting on Website

AHRQ’s Office of Communication (OC) will edit the Final Report. OC will email proposed edits to the EPC authors and copy the TOO. The report authors and OC should aim to complete editing within six weeks of Final Report approval. After editing, OC will work with the AHRQ web team on EHC website posting.

When the Final Report is posted, the EPC the email a link to the Final Report Key Informants and TEP members, thanking them for their participation. The TOO will contact the Partner.

Three months after the Final Report is posted, AHRQ will post the Disposition of Comments document on the EHC website.

https://pra.digital.gov/ https://pra.digital.gov/ https://srdrplus.ahrq.gov/

4. Assist with Translation and Dissemination

AHRQ’s OC may craft a dissemination plan for the evidence report aimed at diverse audiences.

The EPC and the TOO will be asked to provide input on the accuracy and framing of key messages from the final report.

5. Presentations and Abstract Submissions

EPCs are encouraged to present the findings of a report to scientific or non-scientific audiences.

However, (except as required in the Task Order) additional presentations and abstracts are not required deliverables, and reimbursement is not included in the project budget.

Authors may submit abstracts before a report is posted. The EPC must notify the TOO in advance of the planned abstract and conference date, and provide a copy of the abstract and conference information to the TOO when the abstract is submitted. The EPC may present information that is publicly available on the EHC or AHRQ website. For example, abstracts based on the Final Report must be withdrawn if the Final Report posting will occur after the presentation date.

6. Publication of a Manuscript in a Peer- reviewed journal

EPCs are encouraged to publish the findings of project reports in appropriate refereed scientific journals. However, unless specified in the Task Order, a journal manuscript is not a requirement or deliverable, and reimbursement is not included in the project budget. Additionally, the EPC may not publish any manuscripts based on AHRQ-supported work before AHRQ has published the Final Report.

7. Third Party Press Releases

Prior to making any announcement regarding any Task Order award, the EPC shall contact

AHRQ’s Media Relations Office at newsroom@ahrq.hhs.gov. All press announcements will be coordinated with AHRQ and the Department of Health and Human Services.

In addition to AHRQ’s dissemination plans, an EPC’s organization, or the Partner may wish to create its own press release. If so, the organizational contact, report authors, TOO, and OC media relation staff must work together.

Topic Development Brief

C. Topic Development Brief

The goal of Topic Development Brief is to explore and scope a topic or question to inform decisions about further action (e.g., developing a Systematic review or Technical brief; funding further research). The EPC will identify the key decisional dilemma and clinical and/or policy context, evaluate the topic’s fit against established criteria, and summarize this information.

Examples:

• Care Coordination and Care Plans for Transitions Across Care Settings https://effectivehealthcare.ahrq.gov/sites/default/files/related_files/topic-brief1-care-coordination.pdf

• Emergency Medical Service/911 Workforce Mental or Behavioral Health Issues https://effectivehealthcare.ahrq.gov/sites/default/files/wysiwyg/ems-911-workforce-topic-development-brief.pdf

Process

1. Pre-Kickoff call Activities (See Chapter A)

2. Kickoff Call

The EPC will convene a conference call with the TOO and Partner. The purpose is to:

• Discuss purpose of requested project and identify the decisional dilemma(s).

• Describe a plan for developing topic brief.

• Establish lines of communication between the EPC, TOO, other AHRQ staff, and

Partners.

3. Content Expert Input

The EPC must gather input from local clinical experts, the Partner, or other relevant individuals readily available to the EPC. The purpose is to gather background so that topic development is responsive to the decisional needs of end-users.

Discussions are to be limited and efficient. The EPC will use this input to understand aspects of the topic such as practice variation, clinical uncertainty, appropriate comparators, and important subpopulations. For broad nominations, the team may need to focus on areas of greatest importance and relevance to end-users.

Since discussions are limited and for the purpose of background, individuals do not need to submit COI disclosure forms or be approved by the TOO.

4. Draft Topic Development Brief

The EPC will complete a Topic Development Brief. It includes:

https://effectivehealthcare.ahrq.gov/sites/default/files/related_files/topic-brief1-care-coordination.pdf https://effectivehealthcare.ahrq.gov/sites/default/files/related_files/topic-brief1-care-coordination.pdf https://effectivehealthcare.ahrq.gov/sites/default/files/wysiwyg/ems-911-workforce-topic-development-brief.pdf https://effectivehealthcare.ahrq.gov/sites/default/files/wysiwyg/ems-911-workforce-topic-development-brief.pdf

• Summary of topic and decisional dilemma(s), with preliminary Key Questions and

PICOTS

• Preliminary assessment against pre-specified selection criteria

• Proposed disposition

• Considerations for discussion, if any

5. Present Topic Brief to Decision-making Group for Input

The EPC will give a 2-5 minute presentation to summarize findings and considerations from the topic brief to the decision-making group. The decision-making group will discuss findings and ask clarifying questions. The Partner is responsible for convening and managing the decision-making group.

6. Finalize Topic Development Brief

The EPC will finalize the Topic Development Brief with a brief summary of the discussion and rationale for the final disposition, if needed. The TOO will review and approve this document.

The TOO will send the approved Final Topic Development Brief to the Partner after approval.

Schedule Table 1. Sample Topic Development Brief Schedule

Task/Deliverable Format Deliverable Due Date

(from award) Audience Section 508 standards

Upload timeline and staffing plan to Secure

Site. Notify TOO.

Electronic Timeline and staffing plan

1 week AHRQ, TOO

No – (received by TOO)

Upload Staff

Disclosure forms and confidentiality agreements to Secure

Site. Notify TOO.

Electronic Staff disclosure forms and confidentiality agreements

13 days AHRQ, TOO

No – (received by TOO)

Kickoff Call - - 2 weeks - -

Upload summary of call discussion and decisions to Secure

Site. Notify TOO.

Distribute summary to call participants.

Electronic Summary of call discussion

1 week after

Kickoff Call

AHRQ,

TOO

No – (received by TOO)

Draft Topic

Development Brief

Electronic Draft topic development brief

7 weeks AHRQ, TOO

No – (Received by TOO)

Final Topic

Development Brief Electronic Final topic development brief

2 weeks after presentation

AHRQ,

Public

1.1.1 Non-text Content

[A]

1.3.1 Information and

Relationships [A]

1.4.1 Use of Color –

Level A

302.1 Without Vision

302.2 With Limited

Vision

302.3 Without Perception

of Color

Topic Refinement and Systematic Review

D. Topic Refinement and Systematic Review

Most EPC Systematic Reviews include Topic Refinement, and this chapter outlines the Topic

Refinement and systematic review process steps and deliverables.

Topic refinement size will vary by how well the scope has been specified and how many KI will be engaged. This will be specified in the task order. This process is designed to ensure that the research review results in a product that meets the needs of the partner, and is feasibly addressed by a systematic review. Key questions should reflect the uncertainty that decisionmakers, patients, clinicians, and others may have about the topic. Developing clear, unambiguous, and precise key questions is an early and essential step in the development of a meaningful and relevant systematic review.

Systematic review size will vary by the size of the literature base, and number and complexity of the questions. Protocols and systematic review reports are consistent with PRISMA reporting standards.

Process

1. Pre-kick-off call activities (See Chapter A)

2. Kickoff Call

The EPC must lead a conference call with the TOO and Partner, in order to:

• Introduce and discuss role of the Partner

• Discuss Partner’s planned use of the report, dates of associated events, and hard deadlines for EPC deliverables.

• Review decisional dilemmas and the elements of work to be performed, such as any presentations or webinars.

• Review list of proposed Key Informants and Technical Experts. A KI list is not required for projects with small topic refinement.

• Describe a plan to engage Key Informants and Technical Experts

• Establish lines of communication between the EPC, AHRQ, and Partner(s)

• Remind EPC team members of AHRQ’s Privacy and Nondisclosure Policies. EPCs should direct any inquiries from outside persons or organizations to the TOO

• Poll for availability and set up Monthly Call schedule

3. Assemble the Key Informant (KI) panel

After the kick-off call, the EPC must recruit KIs and obtain CVs, signed COI Disclosure forms, and Confidentiality agreements. TOOs can waive the CV requirement for patients or similar participants without readily available CVs. The EPC should advise the KIs that, unless they opt out, the final Systematic Review will acknowledge them by name.

http://www.prisma-statement.org/ http://www.prisma-statement.org/

The PI should review the COI Disclosure forms for potential financial or non-financial conflicts of interest. If a COI is identified, the PI must submit a rationale for including this individual to the TOO.

4. Topic Refinement Document

Topic Refinement focuses the scope, Key Questions and PICOTS so the Systematic Review addresses the key decisional dilemma(s). See Appendix A for guidance.

Before the KI call, the EPC prepares the topic refinement document. This includes the decisional dilemma(s); clarifies the preliminary Key Questions, PICOTS, and logic model; outlines the size and challenges of the literature base; and describes key issues for input from Key Informants.

This is not required for projects with small topic refinement.

5. KI Calls

The EPC must schedule and lead all KI calls. Participants include the TOO, Partner, and the KIs.

The EPC should provide relevant portions of the topic refinement document to KIs at least one week in advance of the call.

KI calls gather input on appropriateness of the Key Questions, PICOTS and logic model to address the identified key decisional dilemma(s). The EPC shall engage KI to further refine the scope of the questions and further clarify the key decisional dilemmas. This process should narrow the scope of the review rather than broaden it.

To meet the scheduling needs of KIs and to address the topic adequately, the EPC may need several calls. The EPC should consider holding the patient or consumer call separately, since patients or consumers may find it difficult to express their views if scientific experts dominate the call.

If the KIs and Partners have differing views, the reason for these differences should be explored and discussed with the TOO. In general, the perspective of the Partner should guide the review, since they will be using the final report.

6. Draft Protocol

The EPC will submit the Draft Protocol to the TOO. The protocol should be developed consistent with PRISMA extension for protocols. The EPC or TOO may request review by a methodologist from another EPC, concurrent with the TOO review.

7. Assemble the Technical Expert Panel (TEP)

The EPC must submit a list of five to nine Technical Experts (TE) to the TOO before TEP engagement. The Pre-Kickoff Call section covers development of the TE list.

The EPC must recruit technical experts and obtain CVs, signed COI Disclosure forms, and

Confidentiality agreements. The EPC should advise the TEP that they will be acknowledged by name in the Systematic Review.

The PI should review the COI Disclosure forms for potential financial or non-financial conflicts of interest. The PI must submit a rationale to include any individual with disclosed COI.

8. TEP Input

The EPC must obtain input from the TEP by email or a call. If a call is scheduled, the EPC must schedule and lead the call. Participants include the EPC, TOO, and the TEP. The EPC should provide the Draft Protocol to participants at least one week before the call, if held. The TEP may provide input on KQs, PICOTS, critical outcomes to be graded, data sources and search strategy, key studies to be included, or minimally important differences for particular outcomes. See Pre-

Kickoff Call Chapter A for additional TEP description.

9. Final Protocol and Posting

The EPC must revise the Draft Protocol to reflect input from the TEP, and submit the Final

Protocol. The EPC will send a copy of the link to the TEP and KIs. The TOO will notify the

Partner of the Protocol posting. The EPC will register the Final Protocol in the PROSPERO database.

Changes to a posted Final Protocol require a Protocol amendment, which summarizes the changes made and the rationale.

10. Supplemental Evidence and Data for Systematic Reviews (SEADS) and Federal

Register Notice

At Final Protocol posting, AHRQ will notify the public about a SEADS portal

(https://effectivehealthcare.ahrq.gov/get-involved/submit-sead) through the EHC listserv. The

SEADS portal is open for 4 weeks.

The decision for a Federal Register notice will be made by the EPC and TOO. Reports that include medical devices, generic drugs, or poorly defined interventions (such as non-drug, health delivery interventions) require Federal Register notice. The end-date of the SEADS portal will be extended to the end-date of the Federal Register notice.

11. Literature Review

The EPC must systematically search, abstract, review and analyze the scientific evidence for each Key Question as outlined in the protocol.

12. Additional TEP Input

During the literature review, the EPC or the TOO may request additional TEP input via a call or group email. The EPC should contact the entire TEP rather than individual members unless very specific expertise is required (e.g., statistician, member is author of a study in question, etc.). The

TEP should not review the data, or be involved in data analyses.

https://effectivehealthcare.ahrq.gov/get-involved/submit-sead

13. Monthly Conference Calls and Summaries

The EPC must summarize progress in monthly conference calls with the TOO. Call frequency may be adjusted, depending on issues that arise, and as requested by the TOO or

EPC. The TOO may request that the agenda include attachments of working documents, such as reference lists, literature retrieval figures, draft summary tables, etc. Within one week after the call, the EPC must submit a summary of the call that describes progress, any problems encountered and their resolution, any foreseeable problems, and any expected change to the delivery schedule.

14. Proposed Peer Reviewer List

Three months prior to the Draft Report due date, the SRC will remind the EPC to upload the proposed peer reviewer list to ScholarOne after approval by the TOO. The EPC will identify potential peer reviewers and discuss them with the TOO and the Partner. The SRC will manage the peer review invitation and collection of COI Disclosure and confidentiality forms through

ScholarOne.

The list should include four first choice Peer Reviewers, eight alternate Peer Reviewers, and 2-3

TEP members. For each individual, the list should include: name, affiliation, perspective and/or expertise, priority (first choice or alternate), role in the review (peer review or TEP), and whether the perspective or expertise is required for the peer review panel.

The SRC will invite peer reviewers and TEP, and copy the TOO. The TOO will review COI disclosures of the reviewers to assemble a balanced peer review panel.

The goal is to confirm three independent and unbiased peer reviewers, not including the TEP, with an appropriate mix of expertise and perspectives.

The SRC will direct individuals not selected as peer reviewers to comment via the EHC website.

15. Preliminary list of included studies and draft PRISMA diagram for systematic review

After full-text review, the EPC shall submit a preliminary list of included studies and draft

PRISMA diagram, with the understanding that changes may occur with data abstraction. This list may be shared with the Partner.

16. Draft Report

The EPC must submit the draft Systematic Review (Draft Report) and appendices via

ScholarOne. An executive summary is not required at this stage. If the Draft report fails to meet a minimal standard, the report will be returned to the EPC and the submission will not be considered as meeting required deadlines. The EPC Director or Associate Director must review the Draft Report and attest to its integrity and quality before submission to AHRQ.

17. Peer Review of Draft Report

The SRC will manage Peer Review through ScholarOne. The goal is to have at least two submitted peer reviews by the close of the Peer Review period. If the required expertise is represented across peer reviewers and TEP reviewers, the TOO should not extend Peer Review unless absolutely necessary.

The SRC will forward unedited Peer Review comments to the EPC as they are received. At the close of Peer Review, the SRC will email all comments to the EPC.

18. Presentation of Draft Report Findings to Partner and TEP

The EPC shall present the findings from the draft report by webinar to the Partner, AHRQ, and

TEP. The EPC should send a copy of materials to TOO before the presentation and send a summary of the presentation discussion to the TOO and call participants after the presentation.

19. Posting of Draft Report for Public Comment

When the Draft Report is posted for Public Review, the EPC should e-mail the Key Informants, and TEP members, thank them for their participation, and invite them to comment on the report via Public comment. The TOO will email the Draft Report link to the Partner.

Public comments are submitted to the EHC website portal. At the end of the Public comment period, all comments will be shared with the EPC.

If anyone contacts the EPC to discuss the report, the EPC should refer the individual to the TOO and request that comments be provided via the public comment mechanism on the EHC site.

20. Literature Update

The EPC must update the literature search during public comment period using the original search strategy to ensure that the search is current (i.e., less than 3 months old when the Final

Report is anticipated to be accepted by AHRQ).

21. Final Report and Disposition of Comments

The EPC may hold a call with the TOO to discuss proposed approaches to address comments from peer reviewers and public. EPCs should document this call in the Author Response document and submit it with the Final Report.

The EPC must submit the Final Report updated with results of the updated search and responses to comments from TEP, peer reviewers and the public. In addition the EPC must submit the an executive summary and abstract, Disposition of peer review and public comments, an Author

Response to AHRQ and Partner comments, PRISMA checklist, and alternate text for all figures.

22. Post-report activities (see Chapter B).

The Task Order Officer (TOO) reviews deliverables before final acceptance. Due dates that fall on a weekend or holiday are due on the next business day.

Table 3: Topic refinement

Task Format Deliverable Due Date

(from award)

Audience Section 508 Standards

Upload timeline and staffing plan to Secure

Site. Notify TOO.

Electronic Timeline and staffing plan

1 week AHRQ, TOO

No – (received by TOO)

Upload Staff Disclosure forms and confidentiality agreements to Secure Site.

Notify TOO.

Electronic Staff

Disclosure forms and confidentiality agreements

13 days AHRQ, TOO

No – (received by TOO)

Upload proposed Key

Informants (KI) and

Technical Expert Panel

(TEP) lists to Secure Site.

Notify TOO.

Electronic Proposed KI and Technical

Expert panel lists

KI list not required for small TR

13 days

(medium or large TR)

AHRQ,

TOO

No – (received by TOO)

Kickoff Call Electronic - 2 weeks - -

Upload summary of call discussion and decisions to Secure Site. Notify

TOO. Distribute summary to call participants.

Electronic Summary of call discussion

1 week after

Kickoff Call

AHRQ,

TOO

No – (received by TOO)

Upload CVs, Disclosure forms and confidentiality agreements for each KI and TEP member to the

Secure Site. Notify TOO.

Electronic CVs, Disclosure forms and confidentiality agreements for each KI and

TEP member

Prior to KI or TEP Call

AHRQ,

TOO

No – (received by TOO)

Upload completed TR

Document to the Secure

Site. Notify TOO.

Electronic TR Document Not required for small TR

3 Weeks

(medium)

5 weeks

(large)

AHRQ,

TOO

No – (TR 1 received by

TOO)

Task Format Deliverable

(from award)

Audience Section 508 Standards

Upload draft protocol to the Secure Site. Notify

TOO.

Electronic Draft Protocol 8 weeks

(small)

11 Weeks

14 weeks

AHRQ,

TOO

No – (Draft received by

TOO)

Table 4: Systematic Review Phase

Task Format Deliverable Due Date

(from award) Audience

Section 508 Standards

For each TE: disclosure form, confidentiality form, and CV. Upload to Secure

Site. Notify TOO.

Electronic Disclosure form, confidentiality form, and CV

Before TEP call

AHRQ,

TOO

No – (received by

TOO)

Upload summary of TEP input and decisions to

Secure Site. Notify TOO.

After approval, distribute summary to TEP and TOO

Electronic Summary of

TEP input

1 week after

TEP input

AHRQ,

TOO

No – (received by

TOO)

Upload Final Protocol and alt text to secure site.

Notify TOO.

Electronic Final protocol 5 weeks AHRQ, Public

1.1.1 Non-text

Content [A]

1.3.1 Information

and Relationships

[A]

1.4.1 Use of Color

– Level A

302.1 Without

Vision

302.2 With

Limited Vision

302.3 Without

Perception of

Color

Submit proposed peer review list to TOO for review and approval.

Upload to Scholar One after approval.

Electronic Peer review list 8 weeks

13 weeks

17 weeks

AHRQ,

TOO

No – (Received by

(from award) Audience

Section 508 Standards

Upload preliminary list of included studies after full-text review and draft

PRISMA diagram to

Secure Site. Notify TOO

Electronic Preliminary included studies list, draft PRISMA diagram

10 weeks

15 weeks

17 weeks

AHRQ,

TOO

No – (Received by

TOO)

Upload summary of additional TEP input and decisions to Secure Site.

Notify TOO. After approval, distribute summary to TEP and TOO

Electronic Summary of

TEP input

1 week after

TEP input

AHRQ,

TOO

Upload Draft Report, alt text spreadsheet, and

PRISMA checklist to

Scholar One. Notify TOO.

Electronic Draft report 24 weeks

31 weeks

35 weeks

AHRQ,

Public

1.1.1 Non-text

Content [A]

1.3.1 Information

and Relationships

[A]

1.4.1 Use of Color

– Level A

302.1 Without

Vision

302.2 With

Limited Vision

302.3 Without

Perception of

Color Submit PPT slides for Partner presentation to COR by email PPT Electronic During peer review

Partner

1.1.1 Non-text

Content [A]

1.3.1 Information

and Relationships

[A]

1.4.1 Use of Color

– Level A

302.1 Without

Vision

302.2 With

Limited Vision

302.3 Without

Perception of

Color

Discuss by conference call peer review or public comments on the Draft

Teleconfer ence

- After peer review and public comment

(from award) Audience

Section 508 Standards

Report with the TOO

(optional).

Upload Final Report, alt text spreadsheet, PRISMA checklist, and disposition of comments document to

Scholar One. Notify TOO.

Electronic Final report 40 weeks

47 weeks

51 weeks

AHRQ,

Public

1.1.1 Non-text

Content [A]

1.3.1 Information

and Relationships

[A]

1.4.1 Use of Color

– Level A

302.1 Without

Vision

302.2 With

Limited Vision

302.3 Without

Perception of

Color

Upload data files to

SRDRPlus in format ready for publication

Electronic SRDRPlus upload

41 weeks

48 weeks

52 weeks

AHRQ,

Public

1.1.1 Non-text

Content [A]

1.3.1 Information

and Relationships

[A]

1.4.1 Use of Color

– Level A

302.1 Without

Vision

302.2 With

Limited Vision

302.3 Without

Perception of

Color

Upload complete Feedback form

Electronic Feedback form 41 weeks

48 weeks

AHRQ,

TOO

No – (Received by

COR, and TOO)

Upload call summary as status report to Secure Site.

Notify TOO.

Secure

Site, notify

TOO

Monthly Monthly Monthly No – (Received by

Standalone Systematic Review

E. Standalone Systematic Review (without Topic Refinement)

A Systematic Review thoroughly assesses and synthesizes research literature to answer Key

Questions that are important to the partner and other end-users.

Process

The steps are the same as TR/SR except that steps 3-5 are omitted.

Schedule

Table 5. Sample Systematic Review Schedule

Task Format Deliverable Due Date

(from award)

Audience Section 508 Standards

Upload timeline and staffing plan to Secure

Site. Notify TOO.

Electronic Timeline and staffing plan

1 week AHRQ, TOO

No – (received by

TOO)

Upload Staff Disclosure forms and confidentiality agreements to Secure

Site. Notify TOO.

Electronic Staff

Disclosure forms and confidentiality agreements

13 days AHRQ, TOO

No – (received by

TOO)

Upload proposed

Technical Experts (TEP) list to Secure Site. Notify

TOO.

Electronic Proposed

Technical

Experts (TEP) list

13 days AHRQ, TOO

No – (received by

TOO)

Kickoff Call - - 2 weeks - -

Upload summary of call discussion and decisions to Secure Site. Notify

TOO. Distribute summary to call participants.

Electronic Summary of call discussion

1 week after

Kickoff

Call

AHRQ,

TOO

No – (received by

TOO)

Upload CVs, Disclosure forms and confidentiality agreements for each TEP member to the Secure

Site. Notify TOO.

Electronic CVs, Disclosure forms and confidentiality agreements for each TEP member

Prior to

TEP Call

AHRQ,

TOO

No – (received by

TOO)

Upload Draft Protocol to secure site. Notify TOO

Electronic Draft protocol 4 weeks AHRQ, TOO No – (Received by

COR, and TOO) Upload summary of TEP input and decisions to

Secure Site. Notify TOO.

After approval distribute

Electronic Summary of

TEP input

1 week after TEP input

AHRQ,

TOO

(from award)

Audience Section 508 Standards summary to TEP and

TOO

Upload Final Protocol and alt text to secure site.

Notify TOO.

Electronic Final protocol 10 weeks AHRQ, Public

1.1.1 Non-text Content

[A]

1.3.1 Information and

Relationships [A]

1.4.1 Use of Color –

Level A

302.1 Without Vision

302.2 With Limited

Vision

302.3 Without

Perception of Color

Submit proposed peer review list to TOO for review and approval.

Upload to Scholar One after approval.

Electronic Proposed peer review list

13 weeks

(small)

20 weeks

(medium)

24 weeks

(large)

AHRQ,

TOO

Upload preliminary list of included studies after full-text review and draft

PRISMA diagram to

Secure Site. Notify TOO

Electronic Preliminary included studies list, draft PRISMA diagram

15 weeks

20 weeks

22 weeks

AHRQ,

TOO

No – (Received by

TOO)

Additional TEP input Teleconference NA Between protocol and draft report

NA NA

Upload summary of TEP input and decisions to

Secure Site. Notify TOO.

After approval distribute summary to TEP and

TOO

Electronic Summary of

TEP Call discussion

1 week after TEP

Call(s)

AHRQ,

TOO

Upload Draft Report, alt text spreadsheet, and

PRISMA checklist to

Scholar One. Notify

TOO.

Electronic Draft report 29 weeks

36 weeks

40 weeks

AHRQ,

Public

1.1.1 Non-text Content

[A]

1.3.1 Information and

Relationships [A]

1.4.1 Use of Color –

Level A

302.1 Without Vision

302.2 With Limited

Vision

(from award)

Audience Section 508 Standards

302.3 Without

Perception of Color

Submit PPT slides for

Partner presentation to COR by email

PPT Electronic During peer review

1.1.1 Non-text Content

[A]

1.3.1 Information and

Relationships [A]

1.4.1 Use of Color –

Level A

302.1 Without Vision

302.2 With Limited

Vision

302.3 Without

Perception of Color

Discuss by conference call peer review or public comments on the Draft

Report with the TOO

(optional).

Teleconference NA After peer review and public comment

NA NA

Upload Final Report, alt text spreadsheet, PRISMA checklist, and disposition of comments document to Scholar One.

Notify TOO.

Electronic Final report 45 weeks

56 weeks

AHRQ,

Public

1.1.1 Non-text Content

[A]

1.3.1 Information and

Relationships [A]

1.4.1 Use of Color –

Level A

302.1 Without Vision

302.2 With Limited

Vision

302.3 Without

Perception of Color

Upload data files to

SRDRPlus in format ready for publication

Electronic SRDRPlus upload

46 weeks

53 weeks

57 weeks

AHRQ,

Public

1.1.1 Non-text Content

[A]

1.3.1 Information and

Relationships [A]

1.4.1 Use of Color –

Level A

302.1 Without Vision

302.2 With Limited

Vision

302.3 Without

Perception of Color

Upload complete

Feedback form

Electronic Feedback form 46 weeks

53 weeks

COR, and TOO)

(from award)

Audience Section 508 Standards

57 weeks

(large)

Upload call summary as status report to Secure

Site. Notify TOO.

Electronic Call minutes Monthly AHRQ, TOO

Living Review

F. Living Review

The purpose of a living review is to continuously report the state of the evidence for end-users. A living review task order will typically include these products:

1. Evidence review (systematic review, technical brief, or rapid review)

2. Living review Surveillance, with a performance period typically of one year

• Annual TEP and protocol amendments, if needed

• Annual presentation to Partner and TEP

• Monthly searches, identification and assessment of new studies, update of conclusions and SOE, and SRDRplus upload

3. Focused update of a living review at specified intervals in the task order

4. Evidence review update at specified intervals in the task order

Example of a Monthly Report:

https://effectivehealthcare.ahrq.gov/sites/default/files/related_files/cannabis-2023-update-surv-rep-2.pdf

Example of a Focused Update of a Living Review:

https://effectivehealthcare.ahrq.gov/sites/default/files/related_files/surveillance-report-1-nonopioid-pharm-chronic-pain.pdf

Process

1. Evidence Review. See relevant evidence review type (systematic review or rapid review)

2. Living Review Surveillance

a. Assemble the Technical Expert Panel (TEP)

The EPC may retain the TEP from the initial systematic review. Once per year the

EPC may recruit new technical experts. Similar to the systematic review process, they must obtain CVs, signed COI Disclosure forms, and Confidentiality agreements from all TEP members. The PI should review the COI Disclosure forms. The PI must submit to the TOO a rationale to include any individual with disclosed COI.

The TOO should approve the COI Disclosure forms (and mitigation plan) within one week. TEPs may not serve until they have been approved by the TOO.

b. Annual TEP input

Once per year, the EPC must obtain input from the TEP either by teleconference or email. If a call is held, the EPC will schedule and lead a TEP call. Participants include the EPC, TOO, and the TEP. The EPC should provide the Protocol to participants at least 1 week before the call. The TEP may provide input on any protocol changes.

https://effectivehealthcare.ahrq.gov/sites/default/files/related_files/cannabis-2023-update-surv-rep-2.pdf https://effectivehealthcare.ahrq.gov/sites/default/files/related_files/cannabis-2023-update-surv-rep-2.pdf https://effectivehealthcare.ahrq.gov/sites/default/files/related_files/surveillance-report-1-nonopioid-pharm-chronic-pain.pdf https://effectivehealthcare.ahrq.gov/sites/default/files/related_files/surveillance-report-1-nonopioid-pharm-chronic-pain.pdf

If no TEP was convened for the initial evidence review the EPC will consult with the

Partner and TOO.

c. Revised Protocol

The EPC shall develop a revised protocol with description of changes to reflect TEP and/or Partner input or details related to the living status of the review. The search strategy should be peer-reviewed if changed significantly. The revised protocol will be posted on the EHC website.

d. Supplemental Evidence and Data for Systematic Reviews (SEADS) and

Federal Register Notice

This the same as TR/SR

e. Literature Scan

The EPC will search the published literature since the last search date up to present using the search strategies in the evidence review protocol. The EPC shall apply inclusion/exclusion criteria as described in the protocol and identify included studies.

f. Monthly report

In a monthly report, the EPC shall submit a list of included studies, PRISMA diagram if relevant, evidence table of new studies, risk of bias of new studies if relevant, and assessment of any change of conclusions. The list of included studies will be a cumulative list over the performance period and shall indicate studies that were added since the previous list. This list may be shared with the Partner.

g. Annual Presentation of Surveillance findings

The EPC shall summarize surveillance findings over the performance period in a presentation to the Partner, TOO, and TEP. The EPC should send a copy of materials to TOO before the presentation and send a summary of the presentation discussion to the TOO and call participants after the presentation.

h. Monthly Update Calls- Same as Step 13, TR/SR

3. Focused Update of a Living Review.

At prespecified intervals during surveillance, based on the list of included studies, AHRQ will exercise the option of a focused update of a living review. If no new studies are identified or the studies do not change conclusions, the option will not be exercised. A part of the review may be updated in the focused update. Elements of a focused update include:

• Brief background to orient reader to the document

• Updated search dates

• Updated PRISMA figure if relevant, evidence table, and ROB table if relevant for new studies

• New synthesis or conclusions including meta-analysis, conclusions, and SOE by key question if relevant

An EPC reviewer and TOO will review the focused update of the living review. After approval, the report will be posted on the EHC website.

4. Evidence Review Update. See relevant product type (systematic review update or rapid review). Timing will specified in the task order.

Schedule

Table 6. Sample Living Review Surveillance Schedule.

Task Format Deliverable Due Date

(from award)

Audience Section 508 Standards

Upload timeline and staffing plan to Secure

Site. Notify TOO.

Electronic Timeline and staffing plan

1 week AHRQ, TOO

No – (received by

TOO)

Upload Staff Disclosure forms and confidentiality agreements to Secure

Site. Notify TOO.

Electronic Staff Disclosure forms and confidentiality agreements

13 days AHRQ, TOO

No – (received by

TOO)

Upload proposed

Technical Experts (TEP) list to Secure Site. Notify

TOO.

Electronic Proposed

Technical

Experts (TEP) list

13 days AHRQ, TOO

No – (received by

TOO)

Kickoff Call - - 2 weeks - -

Upload summary of call discussion and decisions to Secure Site. Notify

TOO. Distribute summary to call participants.

Electronic Summary of call discussion

1 week after

Kickoff

Call

AHRQ,

TOO

No – (received by

TOO)

Upload CVs, Disclosure forms and confidentiality agreements for each TEP member to the Secure

Site. Notify TOO.

Electronic CVs, Disclosure forms and confidentiality agreements for each TEP member

Prior to

TEP Call

AHRQ,

TOO

No – (received by

TOO)

Upload summary of TEP input and decisions to

Secure Site. Notify TOO.

After approval distribute summary to TEP and

TOO

Electronic Summary of

TEP input

1 week after TEP call(s)

AHRQ,

TOO

Upload Revised Protocol and alt text to secure site.

Notify TOO.

Electronic Revised

Protocol

6 weeks AHRQ, Public

1.1.1 Non-text

Content [A]

(from award)

Audience Section 508 Standards

1.3.1 Information

and Relationships

[A]

1.4.1 Use of Color –

Level A

302.1 Without

Vision

302.2 With Limited

Vision

302.3 Without

Perception of Color

Upload monthly report.

Notify TOO

Electronic Monthly report Monthly AHRQ, TOO

No – (Received by

TOO)

Submit PPT slides for

Partner presentation to COR by email

PPT Electronic 4 weeks before end of performance period

1.1.1 Non-text

Content [A]

1.3.1 Information

and Relationships

[A]

1.4.1 Use of Color –

Level A

302.1 Without

Vision

302.2 With Limited

Vision

302.3 Without

Perception of Color

Upload call summary as status report to Secure

Site. Notify TOO.

Electronic Call minutes Monthly AHRQ, TOO

Table 7. Focused update of a living review

Due Date

(from award)

Audience Section 508 Standards

Upload focused update of a living review to the secure site. Notify TOO

Electronic Focused update of a living review report

3 weeks AHRQ, TOO

1.1.1 Non-text

Content [A]

1.3.1 Information

and Relationships

[A]

1.4.1 Use of Color –

Level A

302.1 Without

Vision

302.2 With Limited

Vision

(from award)

Audience Section 508 Standards

302.3 Without

Perception of Color

Systematic Review Update

G. Systematic Review Update

A systematic review update will update the established search for a completed AHRQ systematic review.

The process steps are the same as TR/SR with the following exceptions: Steps 3-5 are not applicable.

7. Draft Protocol - Same as TR/SR, except: The EPC must start with the previous SR’s Final

Protocol, and revise as necessary. The protocol must explain changes and indicate that the report is an update and reference the previous SR.

17. Draft Report- Same as TR/SR, except: The EPC must prepare the Draft Report that updates the previous report, and references the previous report in the background, conclusions, and research gaps sections.

Schedule Same as Standalone SR

Rapid Review

H. Rapid Review

An EPC methods paper by Hartling et al describes different types of Rapid Evidence Products.

The rapid review is intended to provide an assessment of evidence in a compressed timeframe to inform an end-user’s decision using strategic adaptations to systematic review methods and processes. The intent of a Rapid Review is to develop a product that balances quality, rigor, and replicability with utility, transparency, and timeliness. Depending on the intended use of the rapid review, the Partner may have specific requirements, such as the need for risk of bias assessments.

Example:

• Retention Strategies for Medications for Addiction Treatment in Adults With Opioid Use

Disorder

• Rapid Review: Antibody Response Following SARS-CoV-2 Infection and Implications for Immunity: A Rapid Living Review (ahrq.gov)

1. Pre-kick-off call activities same as combined TR/SR, except: KI and TEP lists are not required

2. Kick-off call

The EPC must lead a conference call with the TOO and Partner, in order to:

• Introduce and discuss roles of the Partner

• Review purpose of the product, key decisional dilemmas and the elements of work to be performed, such as any presentations or webinars.

• Discuss the proposed methods for the rapid review

• Establish lines of communication between the EPC, TOO, and Partner.

3. Brief Protocol

The EPC will develop a brief protocol about the project with input from the TOO and Partner. At minimum, it will include the following elements:

• 2-3 sentence background

• Purpose of review

• High-level overview of methods, focusing on key steps that will be streamlined for the rapid review

The EPC will submit this to the TOO. This will be posted on the EHC website.

4. Literature search and analysis

The EPC must systematically search, abstract, review, and analyze the scientific evidence for each Key Question. Consistent with the aim of a rapid evidence product, these steps may be streamlined.

https://effectivehealthcare.ahrq.gov/products/rapid-review-production/white-paper https://effectivehealthcare.ahrq.gov/products/retention-strategies-opioid-use-disorder/rapid-research https://effectivehealthcare.ahrq.gov/products/retention-strategies-opioid-use-disorder/rapid-research https://effectivehealthcare.ahrq.gov/sites/default/files/related_files/rapid-review-immunity-after-covid.pdf https://effectivehealthcare.ahrq.gov/sites/default/files/related_files/rapid-review-immunity-after-covid.pdf

5. Proposed peer reviewer list

This is the same as TR/SR, except:

• The Peer Review list should include three first choice peer reviewers, and two alternate peer reviewers (invited only if first choice reviewers decline)

• The goal is three independent and unbiased peer reviewers who agree to participate

6. Draft report

This is the same as TR/SR

7. Peer Review

Peer Review is the same as TR/SR with the following exceptions:

• EPC reviewer and TOO will do a high-level review prior to peer review, with a full draft report review concurrent with peer review.

• Peer Review will be for one week

• The goal is to have at least one submitted Peer Reviews by the close of Peer Review.

8…