Govt_Response_to_Questions_PSNet.pdf

Government Response to Questions RFP: AHRQ-19-10002 (PSNet) Page 1 of 7

Government Response to Questions:

RFP: AHRQ-19-10002 (PSNet)

Note: “No solicitation amendment necessary” – This response is used when the solicitation as originally written is clearly articulated, when the response would be dependent on the offerors proposed approach, and/or when the Government does not have any additional information to provide.

Technical Questions:

1. Question: Task 1: (Page 9) Is the current site hosted at an AHRQ facility or FedRamp compliant environment?

Government Response: The current website is housed within a Federal Risk and Authorization Management Program (FedRAMP) compliant cloud service provider (CSP) and adheres to the requirements of the Federal Information Security Modernization Act (FISMA) of 2014.

2. Question: Task 1: (Page 9) The SOW states "The website must also be hosted at an AHRQ facility..."

does this imply the contractor will be given access to an AHRQ facility?

Government Response: Please see solicitation amendment.

3. Question: Task 1: What are the underlying technologies (e.g. Linux, Apache, mySql, Php, MS .Net, IIS, MS-Sql) used by the existing site?

Government Response: The PSNet website needs to meet requirements of the Office of Management and Budget (OMB) memorandum M-16-21 “Federal Source Code Policy: Achieving Efficiency, Transparency, and Innovation through Reusable and Open Source Software”. Any technologies that support this requirement are acceptable.

4. Question: Task 1: What is the current technology being used for PSNet?

Government Response: See question 3.

5. Question: Task 1: If migration is part of the contract, how many site pages will be moved to the new PSNet site?

Government Response: No solicitation amendment necessary.

RFP: AHRQ-19-10002 (PSNet) Page 2 of 7

6. Question: Task 1: Develop Web Platform that Houses PSNet (Page 9) Will AHRQ leverage their existing GovCloud instance for hosting, or would the contractor need to maintain and operate our own FISMA compliant hosting environment?

7. Question: Task 1: Develop Web Platform that Houses PSNet (Page 9) Related to content searching and content organization, what is the current taxonomy approach and is AHRQ open to evaluating expanding or refining taxonomy?

8. Question: Task 1: Develop Web Platform that Houses PSNet (Page 9) Is AHRQ or the incumbent using analytics to identify what types of content should be (1) recommended for specific users, (2) what content may or may not be retired and (3) used to help influence or determine other forms of marketing and outreach to PSNet users?

9. Question: Task 1: Develop Web Platform that Houses PSNet (Page 9) On average, how many cases for consideration submitted on a monthly and annual basis?

10. Question: Task 1: Develop Web Platform that Houses PSNet (Page 9): Is the $300 compensation for selected cases paid for by separate budget/contract?

11. Question: Task 1: Develop Web Platform that Houses PSNet (Page 9): Should logged-in users be able to view cases that they have submitted somewhere in the account profile section of the site?

RFP: AHRQ-19-10002 (PSNet) Page 3 of 7

12. Question: Task 1: Develop Web Platform that Houses PSNet (Page 9): Does AHRQ have metrics on the number of users accessing the site? Are there analytics reports available on the pages accessed and frequency?

Government Response:

AHRQ PSNet Month of

August

Total as of 8/31/18

WebM&M cases received 1 959

Total visits 108,314

Total page views 174,256

Avg. daily visits 3,494

Avg. time on site 2 min

Avg. pg. views per visit 2

GovDelivery subscriptions 52,408

CME quizzes passed 412

Primer page views 39,914

PSNet accounts 27,766 Education & Training Catalog page views 623

13. Question: Task 2: Content Development, Production and Coordination (Page 10) Are there specific requirements for hosting video stories and video technologies? Will links to AHRQ's YouTube channel or Vimeo videos be allowed, or other external content housed elsewhere? We are thinking about video stories such as Josie King.

14. Question: Task 2: Is the contractor expected to write articles and content OR just research and provide expert review of existing articles?

15. Question: Task 2.2: (Page 11) The requirement for the PSNet Classics Collection on the PSNet website updates is at least every 3 months or as directed by the COR based on AHRQ’s need / requirements.

Is there flexibility in the frequency of updates, such as monthly or bimonthly?

RFP: AHRQ-19-10002 (PSNet) Page 4 of 7

16. Question: Task(s) – Multiple, (Page - Multiple) Are there established review timelines or service level agreements (SLAs) for the COR approval process? There are many deliverables in the project and understanding the approval process would be helpful in developing a schedule and work plan.

17. Question: Task 2.1: PSNet Collection How are weekly updates to be sent/shared?

18. Question: Task 2.1, 2.2, and 2.3: (Page 10-12) Should all scholarly articles presented on the site be open or public access (e.g., available through PubMed Central, NCBI Bookshelf, Public Library of Science); or, are all relevant article citations desired, regardless of licensing (e.g., licensed university resources behind publisher paywalls)?

19. Question: Task 2.1, 2.2, and 2.3: (Page 10-12) For content queries related to licensed resources, does AHRQ have its own licensed resources to query for content or is it expected that public/open access (e.g., available through PubMed Central, NCBI Bookshelf, Public Library of Science) or other resources be utilized? If the latter, what will be the steps for negotiating potential licensing issues?

Government Response: See question 18.

20. Question: Task 2.4: PSNet Safety Primers What is the expected format of the primers? How long are they expected to be?

21. Question: Task 2.8: (Page 14) Does AHRQ have specific requirements for the peer review process of the Spotlight Cases and Commentaries? Is the expectation that the vendor will recommend a process or will the vendor confirm to an existing peer review process approved by AHRQ?

22. Question: Task 2.9: (Page 14) Does AHRQ have an estimate of the number of CE credits that PSNet users may apply for annually. Is the expectation that the costs for all credits be incurred by the contractor or that some or all of these costs will be incurred by persons requesting the credits?

RFP: AHRQ-19-10002 (PSNet) Page 5 of 7

23. Question: Task 2.9: (Page 14) Please clarify the anticipated workload and potential revenue sources related to providing CE/CME/CEU credits. Specifically, will the contractor have the ability to charge users nominal fees for obtaining CME credits and maintaining appropriate documentation thereof? Is the expectation that the costs for all credits be incurred by the contractor or that some or all of these costs will be incurred by persons requesting the credits? If not, what is the anticipated volume of requests for CE/CME/CEU credits? What is the estimated number of CE credits that PSNet users may apply for annually. Is the contractor expected to install verification tools to support CE/CME/CEU credit requests, such as elapsed time measurements and post-activity quizzes?

Government Response: See question 22.

24. Question: Task 3.1: (Page 15) Is there a defined schedule for releasing enhancements, new features and issue resolution?

25. Question: Task 3.1: (Page 15) Will existing monitoring, security and privacy tools that AHRQ has selected be used in the future? What are the current tools being used?

26. Question: Task 3.2: (Page 16) Are the inquiries required to be maintained in an external tool or database outside of the AHRQ e-mail mailbox? Can a tool or process be recommended to AHRQ?

27. Question: Task 4: Marketing and Promotional Plan (Page 16-17) Can AHRQ provide current user statistics for PSNet, including a breakdown of user types by setting of care, if this information is available?

28. Question: Task 5: Support for Evaluation Activities (Page 18) Please describe the type of external evaluation team that should be used to evaluate the impact of the PSNet content. Can the evaluation be performed by specific individuals and or users or are there other requirements of what the external evaluation team should consist of?

RFP: AHRQ-19-10002 (PSNet) Page 6 of 7

29. Question: Task 5: Support for Evaluation Activities (Page 18) Has there been an evaluation of PSNet already performed and is it publicly available to potential bidders. This evaluation would provide useful background regarding opportunities to improve PSNet.

Government Response: Please see solicitation amendment. The evaluations of PSNet show that overall satisfaction with the site is higher than average and the fixes to improve performance are minor.

30. Question: Task 6: Technical Expert Panel (Page 18) Do panelists receive an honorarium payment on a reoccurring frequency? If yes, what is the amount and frequency?

31. Question: Task 6: Technical Expert Panel (Page 18) Are there existing panelists that should be considered for the new contract year?

32. Question: Task 6: Technical Expert Panel (Page 18) Please clarify the expected size of the required TEP, the expected number of TEP meetings annually, whether all of the meeting will be virtual or how many will be in person, and whether bidders should budget for TEP member travel or honoraria.

33. Question: Section L7: Technical Proposal Instructions (Page 85) We would request that the cover page, table of contents and bibliography be excluded from the 35 page limit

34. Question: Section L7: Technical Proposal Instructions (Page 85) Are there specific requirements or format for the bibliography? Is there specific content or evidence that AHRQ is seeking?

35. Question: Beyond hospitals, what are the intended audiences/user groups for PSNet (e.g., community health center, private practices, public health departments)?

RFP: AHRQ-19-10002 (PSNet) Page 7 of 7

36. Question: How will AHRQ define success on this project? Where and at what level of detail should proposals specify activities for defining and gathering process metrics (e.g., number of clicks or page visits), impact/uptake metrics (e.g., number best/recommended practices implemented), and outcome measures (e.g., reductions in adverse safety events, improvement in safety reporting)?

Business Questions:

1. Question:

Could AHRQ provide an expected level of effort or annual budget for the required work

2. Question:

We understand that a cost reimbursement contract (i.e., cost plus fixed fee) is contemplated for this project. Is AHRQ willing to consider a fixed price contract instead?

Government Response: No solicitation amendment necessary. The Government will award the contract as a cost reimbursement type contract.