AHRQ-21-10006.pdf

Request for Proposal v. FAC 2021-01 Oct 2020 AHRQ-21-10006 Support to Implement Increased Transparency of the Processes of the USPSTF Page 1 of 81

Request for Proposal No. AHRQ-21-10006

PART I - THE SCHEDULE

SECTION A - SOLICITATION FORM

Date Issued: November 25, 2020 Date Questions Due: December 4, 2020 Please Note: All questions and inquires will be processed on a non-attribution basis.

Date Notice of Intent Due: December 17, 2020 Date AHRQ Proposal System Registration: January 7, 2021 12:00 PM, EST Date Proposals Due: January 14, 2021 11:00 AM, EST

You are invited to submit a proposal to the Agency for Healthcare Research and Quality (AHRQ) for Request for Proposal (RFP) No. AHRQ-21-10006 entitled “Support to Implement Increased Transparency of the Processes of the USPSTF”. Your proposal must be developed and submitted in accordance with the requirements and instructions of this RFP.

Solicitation Information: Please review the table of contents and refer to the appropriate section of this RFP for requirements, proposal instructions and evaluation information. The Government intends to award this contract without negotiations, so it is imperative that all questions are addressed during the solicitation phase, and that all proposals submitted include the required information and accept the terms and conditions as stated. Failure to comply with proposal instructions or submit required information may result in the removal of a proposal from consideration for award. Please review Section L carefully.

Proposal Intent: If you intend to submit a proposal in response to this solicitation, please inform the Contracting Officer of your intent by sending an email to Scott Henderson, Contract Specialist/Contracting Officer at scott.henderson@ahrq.hhs.gov no later than December 17, 2020. Subject line shall read: Proposal Intent RFP No. AHRQ-21-10006.

Questions: All Questions regarding this solicitation shall be received in this office no later than the date shown above. All questions shall be submitted electronically by e-mail to Scott Henderson, Contract Specialist/Contracting Officer, at the following email address:

scott.henderson@ahrq.hhs.gov. Subject line shall read: Proposal Questions RFP No. AHRQ- 21-10006. Answers to questions and all solicitation amendments will be posted to the Contract Opportunities web page: https://beta.sam.gov/. It is your responsibility to monitor the web site.

Additional Information:

a. A cost plus fixed fee contract is contemplated for a period of one year, with three one-year options.

b. The Government anticipates awarding one contract from this solicitation on or about June 15, 2021.

https://beta.sam.gov/

Support to Implement Increased Transparency of the Processes of the USPSTF Page 2 of 81

c. Proposals submitted in response to this solicitation must state that they are valid for a period of at least 120 days (reference FAR clause 52.215-1 in Section L of this document.)

d. The NAICS code for this requirement is 541990 with a size standard of $16.5 Million.

Proposal Submission:

To submit a proposal, offerors are required to use the AHRQ Proposal System. Mailed proposal and E-mailed proposals will not be accepted.

In order to access the AHRQ Proposal System offerors must be registered in the system before the proposal due date. To ensure your organization is registered in the system, please send the following information to Scott.Henderson@ahrq.hhs.gov no later than 12:00 PM January 7, 2021:

Official Company name:

DUNS Number:

Point of Contact name, email, phone number

Failure to respond timely and provide the requested information could result in an inability to submit a proposal.

Before the proposal due date, instructions on how to upload proposals to the AHRQ Proposal System will be emailed to your organization’s point of contact.

NOTE: For the Business Proposal submission, the offeror must submit two versions. This includes a Word or PDF version AND an Excel document version.

The proposal shall be signed by an authorized official to bind your organization. Signatures can be electronic if necessary.

System for Award Management: In accordance with Federal Acquisition Regulation, all contractors must be registered in the System for Award Management (SAM) database in order to conduct business with the government [See Section I - FAR clause 52.204-7 System for Award Management].Access the System for Award Management (SAM) at https://www.sam.gov/SAM/.

Reminder: The Procurement Integrity Act prohibits the unauthorized disclosure of, inquiry about, or acceptance of procurement information. Communications with any other individual outside the AHRQ contracting office may result in rejection of an offeror’s proposal. All requests for any information concerning this RFP should be referred to Scott Henderson, (301) 427-1764 or e-mail: scott.henderson@ahrq.hhs.gov. Please note e-mail requests should state subject as RFP

AHRQ-21-10006.

https://www.sam.gov/SAM/

Support to Implement Increased Transparency of the Processes of the USPSTF Page 3 of 81

Table of Contents

PART I - THE SCHEDULE

SECTION A - SOLICITATION FORM

SECTION B - SUPPLIES OR SERVICES AND PRICES/COSTS

SECTION C - DESCRIPTION/SPECIFICATION/WORK STATEMENT

SECTION D - PACKAGING AND MARKING

SECTION E - INSPECTION AND ACCEPTANCE

SECTION F - DELIVERIES OR PERFORMANCE

SECTION G - CONTRACT ADMINISTRATION DATA

SECTION H - SPECIAL CONTRACT REQUIREMENTS

PART II – CONTRACT CLAUSES

SECTION I - CONTRACT CLAUSES

PART III- LIST OF DOCUMENTS, EXHIBITS AND ATTACHMENTS

SECTION J – SOLICITATION LIST OF ATTACHMENTS

PART IV– REPRESENTATIONS AND INSTRUCTIONS

SECTION K - REPRESENTATIONS, CERTIFICATIONS, AND OTHER STATEMENTS OF OFFERORS

SECTION L - INSTRUCTIONS, CONDITIONS AND NOTICES TO OFFERORS

SECTION M- EVALUATION FACTORS FOR AWARD

Support to Implement Increased Transparency of the Processes of the USPSTF Page 4 of 81

SECTION B - SUPPLIES OR SERVICES AND PRICES/COSTS

B.1 BRIEF DESCRIPTION OF SUPPLIES OR SERVICES

“Support to Implement Increased Transparency of the Processes of the USPSTF”. See Section C for a complete description.

B.2 ESTIMATED COST AND FIXED FEE

a. The Estimated Cost (exclusive of fees) for performance of the work under the base period of this contract, including direct and indirect costs is $ To Be Determined (TBD).

b. The Fixed Fee for the base period of this contract is $TBD. The fixed fee shall be paid in installments based on the percentage of completion of work, as determined by the Contracting Officer. Payment shall be subject to the withholding provisions of the Clause ALLOWABLE COST AND PAYMENT and FIXED FEE incorporated herein.

c. The total contract value, represented by the sum of the estimated cost plus the fixed fee for the base period is $TBD.

Period of Performance

Services Estimated Cost

Fixed Fee Total

Base Period Severable $ $ $

B.2 ESTIMATED COST

a. The Estimated Cost for performance of the work under this contract, including direct and indirect costs is $ To Be Determined (TBD).

Period of Performance Services Estimated Cost

Base Period Severable $

B.3 OPTION COSTS

If the Government exercises its option(s) pursuant to the option article in Section H of this contract, the Government’s total contract amount represented by the sum of the estimated cost and fixed fee will be revised as follows:

Period of Performance

Services Estimated Cost Fixed Fee Total

Option Period 1 Severable $ $ $ Option Period 2 Severable $ $ $ Option Period 3 Severable $ $ $

Support to Implement Increased Transparency of the Processes of the USPSTF Page 5 of 81

The base period and each option are separate and distinct from one another for accumulation of costs and billing purposes.

B.4 PROVISIONS APPLICABLE TO DIRECT COSTS

a. Items Unallowable Unless Otherwise Provided

Unless authorized in writing by the Contracting Officer, the costs of the following items or activities shall be unallowable as direct costs:

(1) Acquisition, by purchase or lease, of any interest in real property;

(2) Rearrangement or alteration of facilities;

(3) Purchase or lease of any item of general purpose-office furniture or office equipment regardless of dollar value;

(4) Travel to attend general scientific meetings;

(5) Foreign Travel;

(6) Any costs incurred prior to the contract's effective date;

(7) Rental of meeting rooms;

(8) Any subcontract agreement (including consultants) not otherwise expressly provided for in the contract;

(9) Information Technology hardware or software; and

(10) Food and Beverages.

Note: Food and Beverages are unallowable costs. Contracting Officer Authorization for Food and Beverages will be only made as specifically authorized by statute, which is extremely rare. The contractor shall not provide food or beverages of any type. The contractor shall not accept food or beverages, even if included in, or provided ‘free’ as part of a meeting/conference ‘package’.

b. Travel Costs

This contract is subject to the provisions of Public Law (P.L.) 99-234 which amends the

Office of Federal Procurement Policy Act to provide that contractor costs for travel, including lodging, other subsistence, and incidental expenses, shall be allowable only to the extent that they do not exceed the amount allowed for Federal employees. The Contractor, therefore, shall invoice and be reimbursed for all travel costs in accordance with Federal Acquisition Regulations (FAR) 31.205-46.

Support to Implement Increased Transparency of the Processes of the USPSTF Page 6 of 81

The following applies to Educational Institutions: The Contractor shall invoice and be reimbursed for all travel costs in accordance with OMB Circular A-21 - "Cost Principles for Educational Institutions."

No funding or expenditures will be authorized under this contract for travel or meeting expenses of any person that is not an employee of the contractor or approved subcontractor in accordance with the terms of the contract. See 31 USC § 1345 for appropriations restrictions.

B.5 CONTRACTING OFFICER AUTHORIZATION

Other provisions of this contract notwithstanding, approval of the following items within the limits set forth is hereby granted without further authorization from the Contracting Officer.

To Be Determined

Support to Implement Increased Transparency of the Processes of the USPSTF Page 7 of 81

SECTION C - DESCRIPTION/SPECIFICATION/WORK STATEMENT

Background/References The U.S. Preventive Services Task Force (USPSTF or Task Force) is an independent, volunteer panel of national experts in prevention and evidence-based medicine. Since its inception in 1984, the Task Force has worked to improve the health of all Americans by making evidence-based recommendations about clinical preventive services and health promotion. These recommendations for adults and children address screening tests, counseling about healthful behaviors, and preventive medications.

Since 2009, the Task Force has focused on making its work as transparent as possible so that stakeholders and the public better understand and have more confidence in the approach of the Task Force. This also ensures that its work is open, credible, independent, and unbiased, and is recognized as such. By expanding opportunities for the public and stakeholders to engage in the process, the Task Force believes that its recommendations will be more accurate and relevant.

As a result of these efforts, stakeholders and the public can:

Nominate new members to serve on the Task Force

Nominate new topics for Task Force consideration or request an update of an existing topic

Provide comments on all draft research plans

Provide comments on all draft evidence reports

Provide comments on all draft recommendation statements

For additional information on the Task Force, its procedures including opportunities for public comment, and all documents (e.g., research plans; systematic evidence reviews and recommendation statements), please visit the USPSTF Procedure Manual at http://www.uspreventiveservicestaskforce.org/uspstf08/methods/procmanual.htm.

The USPSTF also has recently redesigned its Web site (www.uspreventiveservicestaskforce.org) to enhance the look and feel, and functionality of the site. The new Web site features an automated system for posting and receiving public comments on draft documents. The redesigned Web site was made available March/April 2020.

The purpose of this contract is to:

1) develop tables for all of the public comments received in response to postings of USPSTF draft research plans, draft systematic evidence reviews, and draft recommendation statements and

2) to aggregate, review, analyze and synthesize public comments received in response to USPSTF draft recommendation statements.

This contract has a base year plus three option years. The work performed in the base year will be the same as the work performed in the option years, if the government decides to exercise the options.

Government Furnished Information

Once awarded, AHRQ will supply the contractor with “read only” access to the content management system (CMS) for the redesigned USPSTF Web site; a weekly calendar of USPSTF topics for release and a template for the citation disposition table and suggested codes (see Task 2.3 below).

http://www.uspreventiveservicestaskforce.org/uspstf08/methods/procmanual.htm http://www.uspreventiveservicestaskforce.org/

Support to Implement Increased Transparency of the Processes of the USPSTF Page 8 of 81

Independently and not as an agent of the Government, the Contractor shall be required to furnish all the necessary services, qualified personnel, material, equipment, and facilities, not otherwise provided by the Government, as needed to perform the Statement of Work as follows.

Under this contract the Contractor shall:

1. Develop tables for up to 30 – 45 postings each year. The tables shall include the public comments received in response to USPSTF draft research plans, draft evidence reports and draft recommendation statements and shall be used by Principal Investigators at Evidence-based Practice Centers, Task Force and AHRQ staff. Draft evidence reports include related reports, such as modeling reports.

2. Establish a methodology for coding the public comments received in response to

USPSTF draft recommendation statements (e.g., develop a coding scheme and process for how coders would be trained, how agreement among coders will be determined, quality control measures, how the data will be analyzed and synthesized).

3. Aggregate, review, analyze and synthesize public comments received in response to

USPSTF draft recommendation statements.

4. Write a report for all draft recommendation statements.

5. Conduct an annual analysis of public comments across topics and write a report summarizing the results.

Statement of Work

Unless otherwise noted under a specific task, this Statement of Work applies to the base period and all options (if exercised).

Task 1: Administration

Subtask 1.1 Kick-Off Call

Within 2 weeks of the task order award, the Contractor shall meet with the Task Order Officer (TOO) and other relevant AHRQ personnel to review and clarify the scope of work and delivery schedule for this task order, to delineate further roles and responsibilities, and to establish communication protocols. This meeting may take place by teleconference or in-person in the base year. The contractor shall provide meeting notes no later than one week after the call.

Subtask 1.2: Quarterly Progress Reports

The Contractor shall submit a quarterly progress report that includes:

Accomplishments during the reporting period Problems or delays the contractor has experienced in the conduct of performance requirements including what specific action is proposed to alleviate the problem(s) Activities planned for the next reporting period, including anticipated staffing requirements, level of effort and cost

Quarterly progress reports will be submitted in electronic format to the COR within ten (10) days after the end of the quarter being reported.

Support to Implement Increased Transparency of the Processes of the USPSTF Page 9 of 81

Subtask 1.3: Monthly Progress Calls

The Contractor shall participate in a monthly progress call with the AHRQ TOO to discuss overall progress, issues related to specific draft products/USPSTF topics, other problems or delays. The monthly calls also may be used for the Contractor to give AHRQ staff a short briefing on the synthesis of public comments on specific USPSTF draft recommendation statements. The Contractor shall provide an agenda at least 3 days prior to the call and a call summary no later than 1 week after the call.

Subtask 1.4: Attend USPSTF Meetings.

The Contractor shall attend 3 USPSTF in-person meetings each year where the USPSTF deliberates upcoming topics. The contractor shall listen to the USPSTF deliberation to inform their understanding of the topics and subsequent public comments.

Subtask 1.5: Prepare Review Across Topics.

The Contractor shall conduct an annual analysis across draft postings and summarize the assessment in a report. Each year, this report shall include a summary of the total number of comments submitted by each USPSTF Partner Organization1 and organizations that frequently submit comments on the research plans, draft evidence reviews, and draft recommendations. Using the established methodology, the contractor shall provide a high-level summary of the nature and themes of the comments.

Task 2: Tables and Reports on Draft USPSTF Documents

Subtask: 2.1: Develop tables of all public comments received in response to USPSTF draft research plans and draft systematic evidence reviews

1. Weekly table of comments for draft research plans and draft systematic evidence reviews.

The Contractor shall develop weekly tables that include all public comments received to date on draft research plans and draft systematic evidence reviews. Public Comments are open for 4 weeks, but may be extended for specific topics.

The Contractor shall have access to the content management system (CMS) of the redesigned USPSTF Web site for aggregating and creating tables of all the public comments received. The CMS will generate a table of the comments that were submitted via the website and the Contractor will use the CMS to produce the weekly table of comments in the format requested by AHRQ. In addition, the Contractor shall add letters received in the mail or email into the table of aggregated public comments. The contractor shall also include comments that are submitted directly to the AHRQ program staff within 2 weeks of the closing date of the comments for the topic.

In the weekly tables, the Contractor shall perform any final formatting needed to meet the agreed upon format. E.g., the Contractor shall provide all the public comments, with a unique ID, question number, type of respondent, and the actual comment. The Contractor shall remove any empty cells that occur as a result of non-response by the commenter, i.e., if a respondent only enters comments in response to questions 1, 3 and 5, the empty cells associated with questions 2, 4 and 6 would be deleted. These tables should be sorted by respondent, i.e., answers to all questions for Respondent 1 would be shown in sequential rows.

1 List of USPSTF Partner Organizations: https://www.uspreventiveservicestaskforce.org/uspstf/about-uspstf/our-partners https://www.uspreventiveservicestaskforce.org/uspstf/about-uspstf/our-partners https://www.uspreventiveservicestaskforce.org/uspstf/about-uspstf/our-partners

Support to Implement Increased Transparency of the Processes of the USPSTF Page 10 of 81

See appendix A and B of the SOW for example of standard questionnaires for the draft research plan and evidence review.

The weekly tables shall be provided on an agreed upon day each week, including a count of the number of comments and estimated due dates of the various final tables.

2. Final table of comments for draft research plans and draft systematic evidence reviews After the public comment period closes, the contractor shall provide final tables that include all public comments received on draft research plans and draft systematic evidence reviews. The Contractor shall provide 3 versions of the final table of all public comments: 1) sorted by respondent, i.e., answers to all questions for Respondent 1 would be shown in sequential rows;

2) sorted by question, i.e., all answers to question 1 would be shown in sequential rows for all respondents; and 3) sorted by question but with all standard answers from the questionnaire removed (version for the Evidence-Based Practice Centers).

In the final tables, the Contractor shall perform any final formatting needed to meet the agreed upon format. E.g., the Contractor shall remove any empty cells that occur as a result of non-response by the commenter, i.e., if a respondent only enters comments in response to questions 1, 3 and 5, the empty cells associated with questions 2, 4 and 6 would be deleted.

The final tables shall be provided no later than 2 weeks after the comment period closes. For topics receiving more than 200 comments, the Contractor shall provide the tables no later than 4 weeks after the comment period closes.

The USPSTF anticipates posting 10 - 15 draft research plans and 10 - 15 draft systematic evidence reviews each year.

Subtask 2.2: Develop Tables and Reports on Draft Recommendation Statements

1. Weekly Table of Comments for Draft Recommendation Statements.

The Contractor shall develop weekly tables for use by the Task Force and AHRQ program staff that include all public comments received up to that date on draft recommendation statements.

The weekly tables shall be due to the TOO on an agreed upon day each week, including a count of the number of comments and estimated due dates of the various final tables. Public Comments are open for 4 weeks, but may be extended for specific topics.

The Contractor shall have access to the content management system (CMS) of the redesigned USPSTF Web site for aggregating and creating tables of all the public comments received. The CMS will generate a table of the comments that were entered via the website and the contractor will use the CMS to produce the weekly table of comments in the format requested by AHRQ. In addition, the Contractor shall add letters received in the mail or email into the table of aggregated public comments. The contractor shall also include comments that are submitted directly to the AHRQ program staff within 2 weeks of the closing date of the comments for the topic.

In the weekly tables, the Contractor shall perform any final formatting needed to meet the agreed upon format. E.g., the Contractor shall provide all the public comments, with a unique ID, question number, type of respondent, and the actual comment. They shall remove any empty cells, i.e.., if a respondent only enters comments in response to questions 1, 3 and 5, the empty cells associated with questions 2, 4 and 6 would be deleted. These tables should be sorted by respondent, i.e., answers to all questions for Respondent 1 would be shown in sequential rows.

See appendix C of the SOW for example of standard questionnaires for the draft recommendation statement.

The weekly tables shall be provided on an agreed upon day each week, including a count of the number of comments and estimated due dates of the various final tables. For select topics or

Support to Implement Increased Transparency of the Processes of the USPSTF Page 11 of 81 topics with a high volume of comments, the contractor shall provide more frequent updates on an agreed upon day each week.

2. Final table of comments for draft recommendation statement. After the public comment period closes, the contractor shall provide final tables that include all public comments received on draft recommendation statements.

In the final tables, the Contractor shall perform any final formatting needed to meet the agreed upon format. E.g., the Contractor shall also remove any empty cells that occur as a result of non-response by the commenter, i.e., if a respondent only enters comments in response to questions 1, 3 and 5, the empty cells associated with questions 2, 4 and 6 would be deleted.

The final tables shall be provided no later than 2 weeks after the comment period closes. For topics receiving more than 200 comments, the Contractor shall provide the tables no later than 4

3. Final Report for Draft Recommendation Statements.

The Contractor shall establish a methodology for coding the public comments received in response to USPSTF draft recommendation statements (e.g., develop a coding scheme and process for how coders would be trained, how agreement among coders will be determined, quality control measures, how the data will be analyzed and synthesized).

The final methodology shall be submitted for approval within the first two week of the start of the contract.

Using this methodology, the Contractor shall review, analyze and synthesize all the comments received during the public comment period for all draft recommendation statements. The Contractor shall write a report for all draft recommendation statements. The type of report will vary depending on the number of public comments received.

1. For draft recommendation statements that receive comments from 25 or fewer respondents, the Contractor shall produce a Summary Report that will include at minimum:

a. Summary of Respondents, Responses, and Themes - the contractor shall provide the total number of comments received, the distribution of comments by respondent type, a list of organizations that provided comments, a list of USPSTF Partner organizations that provided comments, a summary of the number of respondents who agreed or disagreed with the draft recommendation statement, and the themes of the public comments using the approved qualitative methodology. The number of comments to the Draft Evidence Report shall also be provided. The number of suggested references should also be included.

b. Comprehensive Table of Public Comments on Draft Recommendation Statement – final table of all public comments received in the format described above.

c. Reference Disposition Table - a disposition table of all citations for additional studies listed in the public comments that includes: the full citation provided, whether the study was included, excluded or not considered in the systematic evidence review (this information is provided as part of the systematic evidence review) and a disposition for the reference, e.g., excluded due to study design (Letter to the Editor;

Opinion Piece; Clinical Guideline, etc.). AHRQ will provide the Contractor with a template for the citation disposition table and suggested codes.

Support to Implement Increased Transparency of the Processes of the USPSTF Page 12 of 81

The Summary Report shall be provided no later than 2 weeks after the comment period closes.

2. For draft recommendation statements that receive comments from more than 25 respondents, the Contractor shall produce a Full Report, which shall include at minimum:

a. Summary of Respondents, Responses, and References – the contractor shall provide the total number of comments received on the Draft Recommendation Statement, the distribution of comments by respondent type, a list of organizations that provided comments, a list of USPSTF Partner organizations that provided comments, and the nature of the public comments received (e.g., whether respondents agreed/disagreed with the draft recommendation statement), and the number and types of references suggested by respondents. The number of comments to the Draft Evidence Report shall also be provided. The number and nature of suggested references should also be included.

b. Themes Memo (Summary of Findings and Recommendations)– using the approved qualitative methodology, the Contractor shall summarize comments received by describing the themes that emerged in the review of the public comments to the Draft Recommendation Statement. For each theme, the Contractor shall note any variation by respondent type, and highlight any comments that are related specifically to revisions to the draft recommendation statement that would address the themes/major concerns expressed in the public comments.

A Preliminary Themes Memo shall be provided no later than 2 weeks after the comment period closes. For topics receiving more than 200 comments, the Contractor shall provide the memo no later than 4 weeks after the comment period closes.

The Final Themes Memo shall be included in the Final Report.

c. Quotes Table - the contractor shall provide a table that highlights specific comments from respondents that illustrate the major themes/concerns identified during the review and synthesis of all public comments.

d. Methods - the contractor shall provide a section describing the methods for reviewing and synthesizing all the public comments (based on the approved methods). This should include, but is not limited to, how a coding scheme was developed, how coders were trained, how agreement among coders was determined, what quality control measures will be used, and how the data are analyzed and synthesized. The contractor shall also provide the codebook that was used to code the data.

e. Comments of Particular Importance – the contract shall identify and highlight comments that raise salient issues to which the USPSTF might want to give particular attention.

f. Attachments shall include:

o Comprehensive Table of Public Comments for the Draft Recommendation

Statement – final table of public comments to the draft recommendation statement that also includes information about how each response was coded using the established codebook.

o Comprehensive Table of Public Comments for the Draft Evidence Review – final table of public comments to the draft evidence review.

o Draft Recommendation Statement Reference Disposition Table – same as described for the summary table above.

o Draft Evidence Review Reference Disposition Table – same as described for the

Support to Implement Increased Transparency of the Processes of the USPSTF Page 13 of 81 summary table above.

The Full Report shall be provided no later than 4 weeks after the comment period closes. For topics receiving more than 200 comments, the Contractor shall provide the report no later than 6

The USPSTF anticipates posting 10 – 15 Draft Recommendation Statements.

Please note: It is anticipated that the Contractor will have access to sensitive information under this project. All public comments are confidential. The evidence review and recommendation statements are considered sensitive information before they are publicly published by the USPSTF. The Contractor shall guarantee strict confidentiality of the information/data that it is provided during the performance of the contract including the use of non-disclosure agreements with employees and subcontractors. Disclosure of the information/data, in whole or in part, by the Contractor can only be made after the Contractor receives prior written permission in accordance with contract clause H-1. Whenever the Contractor is uncertain with regard to the proper handling of information/data under the contract, the Contractor shall obtain a written determination from the TOO.

Support to Implement Increased Transparency of the Processes of the USPSTF Page 14 of 81

Appendix A – Draft Research Plan Questionnaire Template

Topic Title This opportunity for public comment expires on [DATE].

The U.S. Preventive Services Task Force (USPSTF) welcomes you to the public comments page.

The USPSTF is seeking comments to help make its draft documents more useful to primary care providers and others who are interested.

PLEASE NOTE:

• If you use the "Back" button on your browser or navigate away from this form, you will lose your comments.

• You need to complete your comments in one session. It is not possible for comments to be saved and reopened.

• Open-ended commentary fields in this form have a 4,000 character limit.

Confidentiality/Disclosure Policy (required) Your comments will be handled on a confidential basis. We will not disclose, give, sell, or transfer any personal information about visitors to this Web site unless required by law enforcement or statute. You are not required to provide personal information to comment on this draft.

Acceptance of this Confidentiality/Disclosure Policy is required to submit your comments.

I have read and understand the above Confidentiality/Disclosure Policy

Support to Implement Increased Transparency of the Processes of the USPSTF Page 15 of 81

Section 1

Proposed Analytic Framework

Picture of analytic framework

Please select one of these options.

I agree with it; I have no comments Generally, I agree with it; see comments below

I have concerns; see comments below

I do not wish to give comments on this question

Do you have any comments about the Analytic Framework?

Support to Implement Increased Transparency of the Processes of the USPSTF Page 16 of 81

Section 2

Proposed Key Question # Text of key question

Please select one of these options.

I agree with it; I have no comments

Generally, I agree with it; see comments below

I have concerns; see comments below

Do you have any comments about Proposed Key Question #?

Support to Implement Increased Transparency of the Processes of the USPSTF Page 17 of 81

Section 3

Proposed Contextual Question # Text of contextual question

Please select one of these options.

I agree with it; I have no comments

Generally, I agree with it; see comments below

Do you have any comments about Proposed Contextual Question #?

Support to Implement Increased Transparency of the Processes of the USPSTF Page 18 of 81

Section 4

Proposed Research Approach

Table

Please select one of these options.

I agree with it; I have no comments

Generally, I agree with it; see comments below

Do you have any comments about the Proposed Research Approach?

Support to Implement Increased Transparency of the Processes of the USPSTF Page 19 of 81

Appendix B – Draft Evidence Review Questionnaire Template

Topic Title This opportunity for public comment expires on [DATE].

The U.S. Preventive Services Task Force (USPSTF) welcomes you to the public comments page.

The USPSTF is seeking comments to help make its draft documents more useful to primary care providers and others who are interested.

PLEASE NOTE:

• If you use the "Back" button on your browser or navigate away from this form, you will lose your comments.

• You need to complete your comments in one session. It is not possible for comments to be saved and reopened.

• Open-ended commentary fields in this form have a 4,000 character limit.

Confidentiality/Disclosure Policy (required) Your comments will be handled on a confidential basis. We will not disclose, give, sell, or transfer any personal information about visitors to this Web site unless required by law enforcement or statute. You are not required to provide personal information to comment on this draft.

Acceptance of this Confidentiality/Disclosure Policy is required to submit your comments.

I have read and understand the above Confidentiality/Disclosure Policy

Do you think this report includes all of the relevant studies?

Yes. The report is complete.

No. The report is not complete.

Unsure

Please share citations for the studies you believe should be added.

Support to Implement Increased Transparency of the Processes of the USPSTF Page 20 of 81

Do you agree with how the included studies and the overall evidence have been interpreted?

Yes. The report correctly summarizes the available evidence.

No. The report should be revised.

The report should be revised in the following ways:

What could be done to make the evidence findings in this draft evidence report clearer?

Nothing. The findings of the report are clear.

The findings of report could be made clearer.

The findings of report could be made clearer in the following ways:

Please share anything else about this evidence report that you have not already mentioned.

Support to Implement Increased Transparency of the Processes of the USPSTF Page 21 of 81

Appendix C – Draft Recommendation Statement Questionnaire Template Topic Title This opportunity for public comment expires on [DATE].

The U.S. Preventive Services Task Force (USPSTF) welcomes you to the public comments page.

The USPSTF is seeking comments to help make its draft documents more useful to primary care providers and others who are interested.

PLEASE NOTE:

• If you use the "Back" button on your browser or navigate away from this form, you will lose your comments.

• You need to complete your comments in one session. It is not possible for comments to be saved and reopened.

• Open-ended commentary fields in this form have a 4,000 character limit.

Confidentiality/Disclosure Policy (required) Your comments will be handled on a confidential basis. We will not disclose, give, sell, or transfer any personal information about visitors to this Web site unless required by law enforcement or statute. You are not required to provide personal information to comment on this draft.

Acceptance of this Confidentiality/Disclosure Policy is required to submit your comments.

I have read and understand the above Confidentiality/Disclosure Policy

Based on the evidence presented in this draft Recommendation Statement, do you believe that the USPSTF came to the right conclusions?

Yes; I believe the USPSTF came to the right conclusions

Somewhat; I believe the USPSTF came to the right conclusions in some ways but not in others

No; I do not believe the USPSTF came to the right conclusions

Unsure; I am not sure if the USPSTF came to the right conclusions

Other

Please provide additional evidence or viewpoints that you think should have been considered.

Support to Implement Increased Transparency of the Processes of the USPSTF Page 22 of 81

How could the USPSTF make this draft Recommendation Statement clearer?

What information, if any, did you expect to find in this draft Recommendation Statement that was not included?

What resources or tools could the USPSTF provide that would make this Recommendation Statement more useful to you in its final form?

The USPSTF is committed to understanding the needs and perspectives of the public it serves. Please share any experiences that you think could further inform the USPSTF on this draft Recommendation Statement.

Do you have other comments on this draft Recommendation Statement?

Support to Implement Increased Transparency of the Processes of the USPSTF Page 23 of 81

SECTION D - PACKAGING AND MARKING

All deliverables required under this contract shall be packaged, marked and shipped in accordance with Government specifications. At a minimum, all deliverables shall be marked with the Contract Number, Contractor Name and Item Number. The Contractor shall guarantee that all required materials shall be delivered in immediate usable and acceptable condition. Note as appropriate on the face page of each deliverable, or on the binding, (1) one volume only, or (2) volume 1 of 2, etc.

Support to Implement Increased Transparency of the Processes of the USPSTF Page 24 of 81

SECTION E - INSPECTION AND ACCEPTANCE

E.1 INSPECTION AND ACCEPTANCE

a. The Contracting Officer or the duly authorized representative will perform inspection and acceptance of materials and services to be provided.

b. For the purpose of this SECTION the Contracting Officer’s Representative is the authorized technical representative of the Contracting Officer.

c. Inspection and acceptance will be performed at:

Agency for Healthcare Research and Quality 5600 Fishers Lane Rockville, Maryland 20857

E.2 CLAUSES INCORPORATED BY REFERENCE (FEB 1998)

This contract incorporates the following clause by reference, with the same force and effect as if it were given in full text.

FAR Clause No. Title and Date

52.246-5 Inspection of Services-Cost

Reimbursement (April 1984)

Support to Implement Increased Transparency of the Processes of the USPSTF Page 25 of 81

SECTION F - DELIVERIES or PERFORMANCE

F.1 PERIOD OF PERFORMANCE

a. The base period of performance of this contract shall be from June 15, 2021 through June 14, 2022.

b. If the Government exercises any Options pursuant to the OPTION PROVISION Article in Section H of this contract, the period of performance will be revised as follows:

Option Year Option Period Option Period 1 06/15/2022 – 06/14/2023 Option Period 2 06/15/2023 – 06/14/2024 Option Period 3 06/15/2024 – 06/14/2025

F.2 DELIVERABLE SCHEDULE

The items specified for delivery below are subject to the review and approval of the Contracting Officer’s Representative (COR) before final acceptance. The contractor shall be required to make revisions deemed necessary by the COR. The deliverable schedule may only be changed by a contract modification signed by an AHRQ Contracting Officer. Days mean calendar days.

Unless otherwise noted under a specific task, this Deliverables Schedule applies to the base period and all options (if exercised).

The contractor shall submit the following items electronically to the COR:

Support to Implement Increased Transparency of the Processes of the USPSTF Page 26 of 81

Anticipated Date of Award:

(June 15, 2021)

Task Format Deliverable Due Date Deliverable’s

Intended Audience/Recipient

Applicable Section 508 Standards

Task 1: Administration

Subtask

1.1 Electronic Kick-off Meeting

Notes

No later than one week after the call

COR, Project Staff NA

Subtask 1.2

Electronic

Submit Quarterly Progress Report

No later than 10 business days from the end of the previous quarter

COR, Project Staff NA

1.3

Agenda and summary notes for monthly call

Agenda:

No later than three days prior to the call

Call summary:

no later than one week after the call

COR, Project Staff NA

Subtask

1.5 Electronic Review Across

Topics Report

Draft: no later than 60 days prior to end of contract

Final: no later than 2 weeks prior to end of contract

COR, Project Staff NA

Task 2: Tables and Reports on Draft USPSTF Documents

Support to Implement Increased Transparency of the Processes of the USPSTF Page 27 of 81

Anticipated Date of Award:

(June 15, 2021)

Task Format Deliverable Due Date Deliverable’s

Intended Audience/Recipient

Applicable Section 508 Standards

Subtask

2.1.1 Electronic

Weekly tables for the draft research and draft systematic evidence review comments

First table due one week after the public comment period posted;

subsequent tables due weekly until 2 weeks after the closing date of the comments

COR, Project Staff NA

2.1.2

Final Table for the draft research and draft systematic evidence review comments (3 versions)

Within two weeks of public comment period closing

COR, Project Staff NA

Subtask

2.2.1 Electronic

Weekly tables for the draft recommendation statement

First table due one week after the public comment period posted;

subsequent tables due weekly until 2 weeks after the closing date of the

COR, Project Staff NA

Support to Implement Increased Transparency of the Processes of the USPSTF Page 28 of 81

Anticipated Date of Award:

(June 15, 2021)

Task Format Deliverable Due Date Deliverable’s

Intended Audience/Recipient

Applicable Section 508 Standards

Subtask

2.2.2 Electronic

Final Table for the draft recommendation statement

Within two weeks of public comment period closing for topics receiving <200 comments

Within 4 weeks of public comment period closing for topics receiving >200 comments

COR, Project Staff NA

Subtask

2.2.3 Electronic Final Methodology

Within the first two weeks of the start of the contract

COR, Project Staff NA

Subtask 2.2.3.1

*Electronic Must be compliant

Summary Report <25 Comments

No later than two weeks after the comment period closing

COR, Project Staff, Public

•302 Functional Performance Criteria •602 Support Documentation •603 Support Services •All WCAG A & AA Success Criteria

Support to Implement Increased Transparency of the Processes of the USPSTF Page 29 of 81

Anticipated Date of Award:

(June 15, 2021)

Task Format Deliverable Due Date Deliverable’s

Intended Audience/Recipient

Applicable Section 508 Standards

Subtask

2.2.3.2 Electronic

Preliminary Themes Memo

(for topics with > 25 comments)

No later than two weeks after the comment period closes for less than comments

No later than four weeks after the comment period closes for more than comments

COR, Project Staff NA

Subtask 2.2.3.2

*Electronic Must be compliant

Full Report Recommendation

Statements (for topics with > 25 comments)

No later than four weeks after the comment period closes for less than comments

No later than six weeks after the comment period closes for more than comments

COR, Project Staff, Public

•302 Functional Performance Criteria •602 Support Documentation •603 Support Services •All WCAG A & AA Success Criteria

Revised Section 508 Standards apply to Electronic and Information Technology (EIT) procured, maintained, or used by the Federal Government. The standards were issued under Section 508 of the Rehabilitation Act which requires access to EIT by people with physical, sensory, or cognitive disabilities. This requirement extends to both federal employees and members of the public. Examples of EIT include (but are not limited to) computers, telecommunications equipment, software, websites, and electronic documents. Accessibility is the law and provides for inclusion. Instructions regarding 508 can be found in this contract in Section H; “Electronic

Support to Implement Increased Transparency of the Processes of the USPSTF Page 30 of 81 and Information Technology Accessibility” and “AHRQ Publishing and Communications Guidelines” at https://www.ahrq.gov/policy/electronic/about/policyix.html. Web sites, web tools, and/or web services (any functionality) that are new, redesigned, modified or added cannot go-live until formal approval has been received from AHRQ’s Office of Communications. The Agency will not accept contract deliverables that fail to meet these standards and guidelines, and will require the contractor to perform any tasks needed to revise deliverables to comply with 508 requirements before acceptance.

Electronic content must be accessible to HHS acceptance criteria. Official HHS Accessibility Checklists for various formats are available at https://www.hhs.gov/web/section-508/making-files-accessible/index.html. Electronic Content deliverables should be accompanied by the appropriate checklist or an updated Accessibility Conformance Report.

Copies of any deliverables should be sent to the Contracting Officer upon request.

Agency for Healthcare Research and Quality ATTN: Contracting Officer Division of Contracts Management, OMS 5600 Fishers Lane Mail Stop 06N34B Rockville, Maryland 20857

F.3 CLAUSES INCORPORATED BY REFERENCE (FEB 1998)

This contract incorporates the following clause by reference, with the same force and effect as if they were given in full text. .

FEDERAL ACQUISITION REGULATION (FAR) (48 CFR CHAPTER 1) CLAUSE:

52.242-15, Stop Work Order (AUG 1989) with Alternate I (APRIL 1984).

Support to Implement Increased Transparency of the Processes of the USPSTF Page 31 of 81

SECTION G - CONTRACT ADMINISTRATION DATA

G.1 KEY PERSONNEL, HHSAR 352.237-75 (DEC 2015)

Pursuant to the Key Personnel clause incorporated in Section I of this contract, the following individual(s) is/are considered to be essential to the work being performed hereunder:

NAME TITLE

(To Be Determined)

The clause cited above contains a requirement for review and approval by the Contracting Officer of written requests for a change of Key Personnel reasonably in advance of diverting any of these individuals from this contract. Receipt of written requests at least 30 days prior to a proposed change is considered reasonable.

G.2 CONTRACTING OFFICER’S REPRESENTATIVE

The following Contracting Officer’s Representative will represent the Government for the purpose of this contract:

(TO BE COMPLETED AT TIME OF CONTRACT AWARD)

The Contracting Officer’s Representative is responsible for: (1) monitoring the contractor's technical progress, including the surveillance and assessment of performance and recommending to the contracting officer changes in requirements; (2) interpreting the statement of work and any other technical performance requirements; (3) performing technical evaluation as required; (4) performing technical inspections and acceptances required by this contract; and

(5) assisting in the resolution of technical problems encountered during performance.

The Government may unilaterally change its Contracting Officer’s Representative designation.

G.3 CONTRACTING OFFICER

The Contracting Officer is the only person with authority to act as an agent of the Government under this contract. Only the Contracting Officer has authority to: (1) direct or negotiate any changes in the statement of work; (2) modify or extend the period of performance; (3) change the delivery schedule; (4) authorize reimbursement to the contractor of any costs incurred during the performance of this contract; or (5) otherwise change any terms and conditions of this contract.

Support to Implement Increased Transparency of the Processes of the USPSTF Page 32 of 81

G.4 INVOICE REQUIREMENTS AND INSTRUCTIONS

The directions below must be followed to meet the requirements of a "proper" payment request pursuant to FAR 32.9, and must be in accordance with the General Provisions clause 52.232-25 Prompt Payment (JAN 2017).

a. INVOICE CONTENT REQUIREMENTS

(1) The contractor shall use a Standard Form 1034, Public Voucher for Purchases and Services Other Than Personal, as an invoice coversheet when submitting/depositing invoices. An electronic copy of the SF-1034 in PDF format may be found at https://www.ahrq.gov/funding/contracts/1034-form.html?tca=jWR- LY2xeuTDCg9DluOnvNb131SIoJqIQd06MOL6coI

Instructions how to complete the SF-1034 form can be found at:

https://www.ahrq.gov/funding/contracts/invoice-guide.html?tca=WK1a3t- LpRvys5y_5Vwz2QLqUlfsRvowGd8BVQJqcxM

(2) The Contractor shall furnish the following MINIMUM information as an attachment to the SF-1034 in support of costs submitted:

a. Contractor Information – include organization name and address as it appears on the contract (or task order/BPA call as applicable). Include the DUNS or TIN number

b. Contract Information – include the project number, contract number and if applicable, the task order or BPA call number

c. Invoice Information – include the invoice…