RFP AHRQ-21-10008_Amendment 01.pdf

Request for Proposal v. FAC 2021-02 Dec 2020 AHRQ-21-10008 Amd #1 Patient Safety Organization Privacy Protection Center (PSOPPC) and Network of Patient Safety Databases (NPSD) Systems Page 1 of 112

Amendment No. 01

RFP: AHRQ-21-10008

Project Title: Patient Safety Organization Privacy Protection Center (PSOPPC) and Network of Patient Safety Databases (NPSD) Systems

Table of Changes:

No. Description of Change Section Page No.

1. Added date amendment issued. n/a Pg. 3

2.

Provided clarification language on content management services

1.5 Pg. 13

3.

Provided clarification language on cloud hosting services

3.1 Pg. 18

4.

Provided clarification language on Collaboration site frequency of review

3.4 Pg. 19

5. Clarification regarding the NPSD system 3.5 Pg.19

6. Corrected task numbers from 3.5 to 3.6 3.6 Pg. 20

7.

Provided clarification language on Common Formats development process

4.2 Pg. 21

8.

Provided clarification on Common Formats prioritization

4.2 Pg. 21

9. Provided clarification on Tools used in 4.3 4.3 Pg. 22

10.

Provided clarification on PSOPPC TA ticket volume

10.1.1 Pg. 26

11. Provided clarification on Tracking e-system 10.3.2 Pg. 28

12. Updated reference section from 10.4 to 10.3.2 10.5 Pg. 28

13.

Appendix A added for PSOPPC current system environment

Appendix A Pg. 39-

14.

Updated Section 508 Standards for 10.3.1 and 10.3.2

Section F.2 Deliverable Schedule

Pg. 52

15.

Updated Technical Proposal formatting language

L.6 Pg. 99

16.

Clarified the past performance information section

L.6, L.9 Pg. 99,

17. Increased page limit to 50 pages L.7 Pg. 101

18.

Added Appendix 3 for Preliminary Project Work Plan

L.7 Pg. 101

Databases (NPSD) Systems Page 2 of 112

19.

Updated clarification on Organizational experience

L.7.C and M1.C Pg. 103,

20. Updated clarification on Personnel qualification L.7.D and M.1.D Pg. 103-

21. Clarified past performance information L.8 Pg. 105

22. Clarified/updated sample cost proposal Attachment A Attach A

Changes are indicated by yellow highlighted text in the document below.

Databases (NPSD) Systems Page 3 of 112

Request for Proposal No. AHRQ-21-10008

PART I - THE SCHEDULE

SECTION A - SOLICITATION FORM

Date Issued: Wednesday, December 16, 2020 Date Amendment 01 Issued: Thursday, January 14, 2021 Date Questions Due: Tuesday, January 5, 2021, 10:00AM EST Please Note: All questions and inquires will be processed on a non-attribution basis.

Date Notice of Intent Due: Thursday, January 21, 2021, 10:00AM EST Date AHRQ Proposal Module Registration: Thursday, January 21, 2021, 10:00AM EST Date Proposals Due: Friday, January 29, 2021, 10:00AM EST

You are invited to submit a proposal to the Agency for Healthcare Research and Quality (AHRQ) for Request for Proposal (RFP) No. AHRQ-21-10008 entitled “Patient Safety Organization Privacy Protection Center (PSOPPC) and Network of Patient Safety Databases (NPSD) Systems”. Your proposal must be developed and submitted in accordance with the requirements and instructions of this RFP.

Please Note: This RFP is a total small business set aside. See FAR 52.219-6 and 52.219-14, herein.

Solicitation Information: Please review the table of contents and refer to the appropriate section of this RFP for requirements, proposal instructions and evaluation information. The Government intends to award this contract without negotiations, so it is imperative that all questions are addressed during the solicitation phase, and that all proposals submitted include the required information and accept the terms and conditions as stated. Failure to comply with proposal instructions or submit required information may result in the removal of a proposal from consideration for award. Please review Section L carefully.

Proposal Intent: If you intend to submit a proposal in response to this solicitation, please inform the Contracting Officer of your intent by sending an email to Michelle Beatty, Contract Specialist/Contracting Officer at Michelle.Beatty@ahrq.hhs.gov no later than Thursday, January 21, 2021. Subject line shall read: Proposal Intent RFP No. AHRQ-21-10008.

Questions: All Questions regarding this solicitation shall be received in this office no later than the date shown above. All questions shall be submitted electronically by e-mail to Michelle Beatty, Contract Specialist/Contracting Officer, at the following email address:

Michelle.Beatty@ahrq.hhs.gov. Subject line shall read: Proposal Questions RFP No. AHRQ- 21-10008. Answers to questions and all solicitation amendments will be posted to the Contract

Opportunities web page: https://beta.sam.gov/. It is your responsibility to monitor the web site.

Databases (NPSD) Systems Page 4 of 112

Additional Information:

a. A cost plus award fee contract is contemplated for a period of one year, with four one-year options.

b. The Government anticipates awarding one contract from this solicitation on or about July 28, 2021.

c. Proposals submitted in response to this solicitation must state that they are valid for a period of at least 120 days (reference FAR clause 52.215-1 in Section L of this document.)

d. The NAICS code for this requirement is 541990 with a size standard of $16.5M.

Proposal Submission:

To submit a proposal, offerors are required to use the AHRQ Proposal Module. Mailed proposal and E-mailed proposals will not be accepted.

In order to access the AHRQ Proposal Module offerors must be registered in the system before the proposal due date. To ensure your organization is registered in the system, please send the following information to Michelle.Beatty@ahrq.hhs.gov no later than 10:00 AM Thursday, January 21, 2021:

Official Company name:

DUNS Number:

Point of Contact name, email, phone number

Failure to respond timely and provide the requested information could result in an inability to submit a proposal.

Before the proposal due date, instructions on how to upload proposals to the AHRQ Proposal Module will be emailed to your organization’s point of contact.

NOTE: For the Business Proposal submission, the offeror must submit two versions. This includes a Word or PDF version AND an Excel document version.

The proposal shall be signed by an authorized official to bind your organization. Signatures can be electronic if necessary.

System for Award Management: In accordance with Federal Acquisition Regulation, all contractors must be registered in the System for Award Management (SAM) database in order to conduct business with the government [See Section I - FAR clause 52.204-7 System for Award Management].Access the System for Award Management (SAM) at https://www.sam.gov/SAM/.

Reminder: The Procurement Integrity Act prohibits the unauthorized disclosure of, inquiry about, or acceptance of procurement information. Communications with any other individual outside the AHRQ contracting office may result in rejection of an offeror’s proposal. All requests for any

Databases (NPSD) Systems Page 5 of 112 information concerning this RFP should be referred to Michelle Beatty, (301) 427-1781 or e-mail:

Michelle.Beatty@ahrq.hhs.gov. Please note e-mail requests should state subject as RFP AHRQ- 21-10008.

Databases (NPSD) Systems Page 6 of 112

Table of Contents

PART I ‐ THE SCHEDULE

SECTION A ‐ SOLICITATION FORM

SECTION B ‐ SUPPLIES OR SERVICES AND PRICES/COSTS

SECTION C ‐ DESCRIPTION/SPECIFICATION/WORK STATEMENT

SECTION D ‐ PACKAGING AND MARKING

SECTION E ‐ INSPECTION AND ACCEPTANCE

SECTION F ‐ DELIVERIES OR PERFORMANCE

SECTION G ‐ CONTRACT ADMINISTRATION DATA

SECTION H ‐ SPECIAL CONTRACT REQUIREMENTS

PART II – CONTRACT CLAUSES

SECTION I ‐ CONTRACT CLAUSES

PART III‐ LIST OF DOCUMENTS, EXHIBITS AND ATTACHMENTS

SECTION J – SOLICITATION LIST OF ATTACHMENTS

PART IV– REPRESENTATIONS AND INSTRUCTIONS

SECTION K ‐ REPRESENTATIONS, CERTIFICATIONS, AND OTHER STATEMENTS OF OFFERORS

SECTION L ‐ INSTRUCTIONS, CONDITIONS AND NOTICES TO OFFERORS

SECTION M‐ EVALUATION FACTORS FOR AWARD

Databases (NPSD) Systems Page 7 of 112

SECTION B - SUPPLIES OR SERVICES AND PRICES/COSTS

B.1 BRIEF DESCRIPTION OF SUPPLIES OR SERVICES

“Patient Safety Organization Privacy Protection Center (PSOPPC) and Network of Patient Safety Databases (NPSD) Systems”. See Section C for a complete description.

B.2 ESTIMATED COST, FIXED FEE and AWARD FEE

a. The Estimated Costs (exclusive of fees) for performance of the work under the base period of this contract, including direct and indirect costs is $ To Be Determined (TBD).

b. The Fixed Fee for the base period of this contract is $TBD. The fixed fee shall be paid in installments based on the percentage of completion of work, as determined by the Contracting Officer. Payment shall be subject to the withholding provisions of the Clause ALLOWABLE COST AND PAYMENT and FIXED FEE incorporated herein.

c. The Award Fee Pool that may be earned for the base period is (TBD). Award fee earned shall be based upon an evaluation and determination by the Government in accordance with the procedures detailed in Section H.

d. The total contract value, represented by the sum of the estimated cost, plus the fixed fee and award fee pool for the base period is as follows:

Period of

Performance Services Estimated

Cost Fixed Fee Award Fee

Pool Total

Base Period Severable $ $ $ $

B.3 OPTION COSTS

If the Government exercises its option(s) pursuant to the option article in Section H of this contract, the Government’s total contract amount represented by the sum of the estimated cost, fixed fee, and award fee pool will be revised as follows:

Period of Performance

Services Estimated Cost Fixed Fee Award Fee Pool Total

Option Period 1 Severable $ $ $ $

Option Period 2 Severable $ $ $ $ Option Period 3 Severable $ $ $ $ Option Period 4 Severable $ $ $ $

The base period and each option are separate and distinct from one another for accumulation of costs and billing purposes.

B.4 PROVISIONS APPLICABLE TO DIRECT COSTS

Databases (NPSD) Systems Page 8 of 112

a. Items Unallowable Unless Otherwise Provided

Unless authorized in writing by the Contracting Officer, the costs of the following items or activities shall be unallowable as direct costs:

(1) Acquisition, by purchase or lease, of any interest in real property;

(2) Rearrangement or alteration of facilities;

(3) Purchase or lease of any item of general purpose-office furniture or office equipment regardless of dollar value;

(4) Travel to attend general scientific meetings;

(5) Foreign Travel;

(6) Any costs incurred prior to the contract's effective date;

(7) Rental of meeting rooms;

(8) Any subcontract agreement (including consultants) not otherwise expressly provided for in the contract;

(9) Information Technology hardware or software; and

(10) Food and Beverages.

Note: Food and Beverages are unallowable costs. Contracting Officer Authorization for Food and Beverages will be only made as specifically authorized by statute, which is extremely rare. The contractor shall not provide food or beverages of any type. The contractor shall not accept food or beverages, even if included in, or provided ‘free’ as part of a meeting/conference ‘package’.

b. Travel Costs

This contract is subject to the provisions of Public Law (P.L.) 99-234 which amends the

Office of Federal Procurement Policy Act to provide that contractor costs for travel, including lodging, other subsistence, and incidental expenses, shall be allowable only to the extent that they do not exceed the amount allowed for Federal employees. The Contractor, therefore, shall invoice and be reimbursed for all travel costs in accordance with Federal Acquisition Regulations (FAR) 31.205-46.

The following applies to Educational Institutions: The Contractor shall invoice and be reimbursed for all travel costs in accordance with OMB Circular A-21 - "Cost Principles for Educational Institutions."

No funding or expenditures will be authorized under this contract for travel or meeting expenses of any person that is not an employee of the contractor or approved subcontractor in accordance with the terms of the contract. See 31 USC

Databases (NPSD) Systems Page 9 of 112

§ 1345 for appropriations restrictions.

B.5 CONTRACTING OFFICER AUTHORIZATION

Other provisions of this contract notwithstanding, approval of the following items within the limits set forth is hereby granted without further authorization from the Contracting Officer.

To Be Determined

Databases (NPSD) Systems Page 10 of 112

SECTION C - DESCRIPTION/SPECIFICATION/WORK STATEMENT

Independently and not as an agent of the Government, the Contractor shall be required to furnish all the necessary services, qualified personnel, material, equipment, and facilities, not otherwise provided by the Government, as needed to perform the Statement of Work as follows:

Statement of Work

Unless otherwise noted under a specific task, this Statement of Work applies to the base period and all options (if exercised).

NOTE: HIPAA applies to this Statement of Work.

Introduction/Background

The Agency for Healthcare Research and Quality (AHRQ) implements and administers several aspects of the Patient Safety and Quality Improvement Act of 2005 (Patient Safety Act). The Patient Safety Act and its regulations (the Patient Safety Rule) authorized the creation of Patient Safety Organizations (PSOs), which collect, aggregate, and analyze confidential and privileged information regarding patient safety and the quality of healthcare. The goals of the Patient Safety Act are: to encourage the expansion of voluntary, provider-driven initiatives to improve patient safety and healthcare quality to promote rapid learning about the underlying causes of risks and harms in the delivery of health care; and, to share those findings widely, thus, speeding the pace of improvement. The Patient Safety Act also authorizes AHRQ to develop and maintain a Network of Patient Safety Databases (NPSD), to which PSOs and others can voluntarily contribute non-identifiable patient safety event information, termed “patient safety work product (PSWP).”1 By law, the NPSD can only receive and report on non-identifiable PSWP. The information in the NPSD shall be used to analyze national and regional statistics, including trends and patterns of healthcare errors, and the results of such analyses will be publicly available and also included in AHRQ’s annual National Healthcare Quality and Disparities Report (QDR). AHRQ understands the complexity of ensuring that information submitted to the NPSD is non-identifiable and the costs associated with adherence to the confidentiality provisions of the Patient Safety Rule. Therefore, AHRQ established the PSO Privacy Protection Center (PSOPPC) to assist PSOs and others with meeting the statutory requirement that PSWP be made non-identifiable prior to sending to the NPSD. The PSOPPC will receive patient safety information in a standardized format (AHRQ Common Formats) and render it non-identifiable before transmitting it safely and efficiently to the NPSD.

The Patient Safety Rule requires PSOs, to the extent practical and appropriate, to collect patient safety event data from providers in a standardized manner. AHRQ’s common definitions, data elements, and reporting formats are known as the Common Formats. Currently, there are two types of Common Formats, Common Formats for Event Reporting and for Surveillance. The PSOPPC only accepts data in AHRQ’s Common Formats. The PSOPPC also delivers technical assistance to PSOs, to include how to use AHRQ’s Common Formats.

Databases (NPSD) Systems Page 11 of 112

1 Additional details about the Patient Safety Act and Patient Safety Rule can be found at www.pso.ahrq.gov. PSWP is defined in the Patient Safety Act and Rule.

The Common Formats include event descriptions, specifications for patient safety aggregate reports, delineation of data elements to be collected for different types of events, and technical specifications for electronic data collection and reporting. The technical specifications promote standardization of collected patient safety event information by stipulating rules for data collection and submission. The PSOPPC maintains the Common Formats and the technical specifications. The Common Formats and the associated technical specifications are available on the PSOPPC Web site (www.psoppc.org).

Scope of Work Requirements Note: EDOC = Effective Date of Contract OR Option Exercise

The PSOPPC System is maintained by the Contractor. PSOs submit data to the PSOPPC, where the Contractor ensures that the data is non-identifiable before submitting it to the NPSD System maintained by AHRQ. The PSOPPC and NPSD (known as the “Systems”) are built around these guiding principles:

a. All information received by the Systems shall be handled with extreme care to ensure its confidentiality and security, particularly Patient Safety Work Product (PSWP), which may contain Protected Health Information (PHI) under the Health Information Portability and Accountability Act (HIPAA).

Refer to Attachment 1 for more details about the Information Security and Privacy Requirements.

b. All Systems processes shall adhere to high standards of quality control.

c. The Systems shall be managed efficiently and transparently.

d. The PSOPPC System shall operate in tight coordination with the NPSD

System, the ultimate recipient of non-identifiable information processed by the PSOPPC contractor, and other stakeholders (where applicable).

e. The enterprise development/solution, architecture design, data modeling, ease of migration (e.g., cloud based), shall be adaptable/capable to receive large volumes of records from any database system and shall be robust and agile and meet industry standards.

f. Infrastructure between the PSOPPC and the NPSD systems should not be shared in any way that will jeopardize confidentiality of PSWP, risk integrity of the required firewalls between the PSOPPC and NPSD systems, or permit the NPSD users to re-identify providers/patients nor to access identifiable information at the PSOPPC System.

1. Maintain the PSOPPC and PSO Collaboration Community Systems

1.1. Confidentiality and Security Protections: The most critical aim of the Systems is to protect the security and confidentiality of any PSWP in accordance with the Patient Safety Rule. Please see the attached Attachment 1 Information Security and Privacy Requirements. The Contractor shall also take other steps that are deemed necessary by the COR and AHRQ Security Officer (SO) to adequately protect data confidentiality and security. (See more information on the Patient Safety Rule and Act at www.pso.ahrq.gov). These may reflect any potential new guidelines or requirements that HHS implements

Databases (NPSD) Systems Page 12 of 112 subsequent to award of the contract. The Contractor shall present the critical milestones in the project management timeline to the COR two

(2) weeks after EDOC. As needed, when new/upgraded software(s) and/or changes in requirements are made, the Contractor shall review, update, and evaluate critical milestones in the project management documents every week. The Contractor shall draft and finalize project management documents within two (2) weeks after receiving comments from AHRQ. Please further refer to Attachment 1.

1.2. Security and Privacy Requirements: The Contractor shall be responsible for the security and the privacy of the system. The PSOPPC System has been determined to be a Federal Information Processing Standards (FIPS) category level moderate. The Contractor shall be responsible for complying with all security and privacy requirements set forth in Attachment 1.

1.3. Maintain Systems Security Plan and other requirements: The Contractor shall identify to the AHRQ Security Officer (SO) all known security and privacy requirements to include reporting of Risk Analyses, Plan of Action and Milestones (P.O.A.&M) completion, and a discussion that imparts a clear understanding to the AHRQ SO of the Risk Profile (including Residual Risk) of the System.

The Contractor shall develop, maintain, review and update the

Nessus report on a monthly basis.

The Contractor shall conduct and maintain a PIA, as defined by

Section 208 of the E-Government Act of 2002 and Federal Acquisition Regulation (FAR) Clause 52.239-1, and required by HHS policy. The PIA shall be completed in accordance with HHS PIA guidance (http://www.hhs.gov/ocio/securityprivacy/privacyresources/pias.

html). Periodic reviews, no less than annually, shall be conducted to determine if a major change to the system has occurred, and if a PIA update is subsequently required. If it is determined that an update is necessary, the Contractor shall make the necessary changes to the PIA.

1.4. The Contractor shall submit the Security Controls Assessment and Contingency Plan Test Report on an annual basis. The Contractor shall maintain a maintenance log available to the COR on a monthly basis. Please further refer to Attachment 1.

1.5. Software Engineering: The Contractor shall establish and adhere to a high level of software development, testing, documentation and quality control standards and procedures. The Contractor shall deliver documentation of the proposed procedures and tools to be used for these functions and review the procedures and tools with the COR for approval.

Prior to final closeout and accounting, the Contractor shall create a schedule for trimming production programs, logs, listings, and format programs as part of the final close-out procedures. The Contractor shall

Databases (NPSD) Systems Page 13 of 112 deliver documentation of all proposed hardware, software, security, backup/recovery, networking and other Information Technology (IT) infrastructure components and solutions needed to support all contractual efforts and shall obtain approval of the proposed solutions from the COR.

The Contractor shall also prepare responses to periodic HHS reporting requirements for accounting for information technology (IT) activities within the Systems. For example, HHS requires that projects with an IT component distinguish between those costs which are Development / Modernization/Enhancement (DME) and those which are Steady State (SS) and report annually on these expenditures. The Contractor shall provide support in assessing and reporting such information. The Contractor shall use an industry standard, object-oriented software design and software technology solutions, three (3)-tier architecture, and enterprise reusable object software components to ensure cost-effective system development, operations and maintenance. The contractor shall maintain all project documentation and deliverables in a software repository consistent with AHRQ’s approach to configuration management and project tracking. The Contractor shall use commercial off-the-shelf (COTS) components when possible. The contractor shall use content management services for managing contents. The Contractor shall adhere to the documentation standards and development guidelines contained in Attachment 1 for all developed and deployed software products. The delivery schedule for documentation deliverables shall be identified in the Project Work Plan (PWP) and shall align with the Delivery Schedule in Section F. For all of the above deliverables for this subtask, the Contractor shall review, update, and evaluate documents annually, and as needed due to changes. The Contractor shall also draft documents within two (2) months after EDOC. The Contractor shall finalize documents after receiving comments from AHRQ within two (2) weeks.

2. Project Management.

2.1. Project Management: The Contractor shall maintain methods to ensure that project tasks and subtasks are coordinated, aligned, and streamlined so that work progresses in an orderly, timely, and efficient manner (e.g., Integrated Master Schedule (IMS)). The Contractor shall maintain an agile process for planning future project activities so that tasks are completed within expected timeframes and budgets. The Contractor shall use quality control procedures to monitor and evaluate the quality of the products that will be delivered. The Contractor shall document all proposed project management methods and software tools for review by the COR. In addition, the Contractor shall provide the COR with documentation of the changes made to project plans and reports. The Contractor shall present the IMS timeline to the COR two

(2) weeks after EDOC. The Contractor shall draft changes to the IMS one (1) week or sooner after changes to the timeline are made and finalize after receiving comments from AHRQ in two (2) weeks upon COR approval. The Contractor shall notify COR immediately if there are any changes via email/phone call and follow up with

Databases (NPSD) Systems Page 14 of 112 documentation.

2.1.1. The Contractor shall identify activities that would require documentation such as backlogs, current activities, action, and validation. The Contractor shall develop, maintain/update, and document the issues identified in the Activities log and review the log on a weekly basis with COR.

2.2. Contract kick-off meeting: The Contractor shall schedule a conference call line and video/webinar link, and coordinate with the COR for a meeting room if needed. The Contractor shall develop an agenda and slides. The Contractor shall meet with the AHRQ team within one (1) week of contract award. The Contractor shall send out meeting information and meeting materials (e.g., agenda, slides) at least 1 day prior to the meeting. The Contractor shall take notes during the meeting and submit the meeting minutes 2 days following the contract kick-off meeting.

2.3. Strategy Meeting with AHRQ: The Contractor shall hold in person meetings, if possible, with AHRQ for a project review and strategy session twice a year at the AHRQ office in Rockville, Maryland.

Starting from OY1: The contractor shall actively engage in conversations with AHRQ to develop high-level strategic goals for the upcoming OY. From these discussions, the Contractor shall produce a concrete work plan that integrates the different tasks under this contract and how they work together. Major tasks that align with AHRQ’s strategic goals will be identified, priorities assigned, and high level time line will be developed and discussed. The Contractor shall coordinate the meeting with the COR by scheduling a meeting room, call in number and video/webinar link, agenda and provide minutes (and, if applicable, develop an Action log) after the meeting. The Contractor shall send out meeting information and meeting materials (e.g., agenda, slides) at least 3 days prior to the meeting. The Contractor shall take notes during the meeting and send out minutes 3 days after the meeting.

2.4. Participate in Weekly Conference (Phone, video, webinar) Meetings with the COR: The Contractor shall participate in formal conference meetings (Phone, video, webinar) with the COR and other staff as assigned, as frequently as once a week, to review progress, identify problems and discuss possible solutions to problems. In advance of the conference meeting, the Contractor shall prepare and send an agenda, which highlights key activities and any problems encountered.

The Contractor shall take notes and deliver minutes (and, if applicable, develop an Action log) after the meeting (3 days after the meeting) to the COR. It is anticipated that less formal communications will also occur on a more frequent basis, as needed with the COR, and other AHRQ staff as assigned. The Contractor shall send out meeting information and meeting materials (e.g., agenda, slides, call-in number, video/webinar link) at least 1 day prior to the meeting.

Databases (NPSD) Systems Page 15 of 112

2.5. Develop Draft PWP and Use Project Management Industry Best

Practices and Earned Value Management (EVM) System: The Contractor shall report earned value monthly based on actual data from (or reconcilable with) its accounting systems. The Government and the Contractor shall negotiate the level of the Work Breakdown Structure (WBS) at which earned value shall be reported. If subcontractors are used, the Contractor shall develop and implement a system to ensure that its subcontractors comply with this requirement, and must address performance at the prime and subcontractor level. The Contractor shall deliver with the PWP, a contract-level IMS that rolls up all time-phased WBSs for the individual tasks. This IMS shall include the dependencies that exist between tasks. The Contractor shall deliver, review and update the PWP and Critical Paths at the contract-level.

2.5.1. The Contractor shall report the following Earned Value

Management (EVM) data for the entire project and specific deliverables/milestones in the monthly report using an AHRQ customized Contract Performance Report (CPR) format that will be provided after award:

a. Planned Percent Complete.

b. Actual Percent Complete.

c. Budget at Completion (BAC) – total planned cost at project completion.

d. Budgeted Cost of Work Scheduled (BCWS) – cumulative expected (planned) cost of work scheduled to be completed by a given period.

e. Actual Cost of Work Performed (ACWP) – cumulative amount actually spent for completed activities at a given period.

f. Budgeted Cost of Work Performed (BCWP) – cumulative planned cost of work completed (the “earned value”).

g. Cost Variance (%) (CV) – the difference between the actual cost and the work performed converted into a percentage.

h. Schedule Variance (%) (SV) – the variance between the actual and planned scheduled converted into a percentage.

i. Cost Performance Index (CPI): a measure of the conformance of the actual work completed to the actual cost incurred.

j. Schedule Performance Index (SPI): a measure of the conformance of actual progress to the planned progress.

2.5.2. The Contractor shall provide a corrective action plan with the monthly earned value report in cases where cost or schedule, as measured by Cost Performance Index (CPI) and Schedule Performance Index (SPI), exceeds the established performance

Databases (NPSD) Systems Page 16 of 112 thresholds. The Government and the Contractor shall negotiate the performance thresholds. All IT related performance thresholds shall not exceed the department’s standard of plus or minus 10% or a corrective action plan needs to be submitted. The corrective action plan shall address the following items:

a. Brief description of the issue (identifies the nature of the issue as cost, schedule, performance or resources and explains why the issue came to be).

b. Impact on project completion.

c. Brief description of the corrective action plan strategy.

d. Corrective tasks/actions that are required to eliminate or mitigate the issue, including task/action number, point of contact, start and end dates and comments.

2.5.3. If the corrective action plan requires re-baselining (the performance baseline), the corrective action plan also shall address the following items:

a. The proposed new baseline.

b. The reason(s) why re-baselining is required.

c. Documentation of the reasonableness of the proposed new baselines and the accuracy of any new cost estimates.

The Performance Baseline, WBS, BCWS, Integrated Master Schedule, and monthly earned value report shall clearly breakout the deliverables and activities related to the development and maintenance of systems, software, databases, and Web sites. It shall also clearly separate DME deliverables and activities from operations and maintenance deliverables and activities.

2.5.4. Monthly EVM reports shall be submitted to the COR (and other

AHRQ staff as designated by the COR) by the 10th calendar day after the end of the reporting month. If the tenth (10th) calendar day falls on a weekend or holiday, the EVM report will be submitted the Friday before the weekend or holiday. If the final EVM report is not completed by the tenth (10th) calendar day, an estimate can be filed until the final monthly report is completed. The Contractor shall notify the COR if an estimate will be filed.

2.5.5. Revised/Final PWP: After receiving feedback from AHRQ on the

PWP draft, the Contractor shall deliver a revised overall PWP that details the life cycle of the project, including phases, iterations, and key milestones including integration points and procurement events, scope, cost, schedule, risks, and quality assurance for all systems and software. The draft (updated from the proposal submission) PWP shall be due two (2) weeks after EDOC.

Subsequent PWPs will be due two (2) weeks after the Contractor receives AHRQ’s comments on the draft. The Contractor shall

Databases (NPSD) Systems Page 17 of 112 review and update on a weekly basis.

2.6. Prepare Quarterly Project Progress Reports: On a quarterly basis, the Contractor shall submit to the COR written reports describing major activities of the project, beginning in the second quarter following the effective date. Each progress report shall list, by major task, project activities of the past quarter, the cost of those activities, the anticipated next quarter's high-level activities, problems encountered and proposed solutions, milestones, and any other information which has a significant impact on ongoing or planned activities or costs.

Costs shall be reported by sub-tasks as well. The report shall also compare progress and resource expenditures to the original schedule and budget and provide explanations for any variances, assess whether the current total estimated contract cost is sufficient to complete the contract. The quarterly progress report shall be delivered to the COR no later than ten (10) calendar days after the end of the reporting quarters (first and third quarters). The last quarter report shall be incorporated in the Annual report. If there are delays or issues identified by the COR, the Contractor shall provide monthly reports (available no later than ten calendar days after the end of the reporting month) instead of quarterly reports.

2.7 Develop Leadership Dashboard

The Contractor shall develop leadership dashboards that allow the tracking of the most important executive metrics across all tasks in this Contract, including selected metrics from the PSOPPC/NPSD analysis and PSO Profile submissions and review. The contractor shall participate in the initial metric selection and recommend additional specific metrics and/or indicators, such as those residing within existing Google analytics reports of PSOPPC website. The leadership dashboard shall be dynamic, with information continuously updated and available to AHRQ.

2.8. Prepare Annual Report

The Contractor shall submit an annual report summarizing the work accomplished in the contract at the end of each contract year (completion of option years, if exercised). This report shall include a description of the project overview, a description of all processes used to generate process information received by the Systems, documentation, and summary of all major deliverables during the report year. The annual report shall also include an estimate of “cost-to-complete” showing major tasks in order to evaluate resources for that year’s activities. This shall be submitted as part of Annual Report.

The report is due within thirty (30) days before end of each contract year. The Contractor shall not have to provide an annual report in the final year of the contract because a final report will be available (see Task 2.9).

2.8.1. The Contractor shall provide zip files (documents in native

Databases (NPSD) Systems Page 18 of 112 formats) for all deliverables for each performance year in an e-file and hard disk. The Contractor shall submit the zip files when the annual report (or, for the last contract year, when the final report described in 2.9) is submitted. The files are due within thirty (30) days before end of each contract year.

2.9 Prepare Final Report: The Contractor shall submit a final report summarizing the work accomplished in the contract at the end of the last contract year (or completion of option years, if exercised). This report shall include a description of the project overview, a description of all processes used to generate process information received by the Systems, documentation, and summary of all major deliverables under the contract, user support, dissemination, recommendations for future efforts, and other issues as determined in conjunction with the COR.

The report is due within thirty (30) days before end of the last contract year.

3. Web Sites and Updated/Refreshed Web Sites Releases

3.1. Both maintenance of the current Web sites and development of new content language and functionality must follow AHRQ’s information systems security and privacy requirements (See Attachment 1). In addition, all aspects of the Web sites shall meet 508 compliance. The PSOPPC Web site is resident in a separate AWS environment from AHRQ’s AWS enclave, and the Contractor shall coordinate all relevant IT activities with AHRQ to provide these services, as necessary, with each release and update. The Contractor shall review Web sites content and web functionality annually and update as necessary. The Contractor shall submit materials four (4) weeks before each release and finalize two (2) weeks after receiving comments from AHRQ upon COR approval.

Note: The contractor is expected to cover all cloud hosting costs for the PSOPPC.

3.2. The Contractor shall maintain and support the online system for PSOs to complete AHRQ’s PSO Profile Form via the secure PSOPPC portal. Forms are submitted on an annual basis. The Contractor shall maintain/update the PSO Profile Form records. Annual aggregate data from the PSO Profile Form shall be prepared for AHRQ quantitatively and graphically using a Data Analytic Interactive Dashboard (e.g., Tableau or similar COTS tool). The contractor shall work with the COR to maintain/update the existing Data Analytic interactive Dashboard and update the layout/content/ graphics of the dashboard as AHRQ’s information, and visualization priorities evolve.

The Contractor shall update the Data Analytic Interactive Dashboard with new PSO Profile information on an annual basis. The Contractor shall also prepare materials, including slides, presenting information gathered through AHRQ’s PSO Profile Form for use during the PSO Annual Meeting and for other purposes, as requested by AHRQ. All

Databases (NPSD) Systems Page 19 of 112 other activities related to the PSO Annual Meeting will be conducted separately and are not a part of this Statement of Work.

3.3. Standard Reports: The Contractor shall maintain functionality of the Web site that allows standard reports to be generated from PSO data submission. This includes management dashboard data submission (MDDS), the PSO profile dashboard, google analytics reports, and comparative national dashboards. In addition, the system allows PSOs to run their own aggregate reports on patient safety event data submitted (clinical dashboards). The Systems use the report and technical specifications developed for AHRQ’s Common Formats to build and maintain this functionality for PSOs. The Contractor shall follow the requirements of the Patient Safety Rule and security measures that prohibit PSOs from seeing other PSO’s data without authorization from the other PSO. Full details on the Patient Safety Rule can be found via AHRQ’s PSO Web site:

https://pso.ahrq.gov/legislation. The Contractor shall review and update the Standard Reports on a periodic basis (e.g., annual).

3.4. Maintain and Update/Upgrade/Refresh for PSOPPC and Collaboration

Community Systems: The Contractor shall develop and implement upgrades to the Web sites to reflect additional data submissions as well as revisions of the Common Formats, the PSO Profile Form, and any other relevant content. The Contractor shall add functionality to the web sites when deemed appropriate to reflect the activities of the Systems, with approval from the COR. The Contractor shall develop protocol(s) to address systems’ functionalities and log issues and resolutions. The Contractor shall conduct procedures for update/upgrade/refresh purposes. The Contractor shall schedule a periodic (2 times a year) review and update of the protocols. The contractor shall prepare and submit to the COR a first draft of the protocols within 3 months of the EDOC. The Contractor shall finalize the protocol four (4) weeks after receiving comments from AHRQ. The Contractor shall maintain an update/upgrade/refresh log of the Web sites, available to COR on a monthly basis.

3.5. Transition NPSD historical contents (Base Year only):

Note: The government hosts the NPSD system and does all maintenance and updates.

The Contractor shall receive non-identifiable information (e.g., data embedded in table shells referenced in Task 12) in the current NPSD System.

3.5.1 Receive and Review Materials on the NPSD System.

The Contractor shall receive and review materials produced under the prior NPSD contract from AHRQ Within 14 days of EDOC, AHRQ shall deliver these materials in electronic format to enable the Contractor to continue current analytic work quickly and

Databases (NPSD) Systems Page 20 of 112 efficiently.

These materials shall include:

a. Information on the PSQIA and related regulations.

b. Documents related to PSOs and the Systems.

c. NPSD PSOPPC system architecture.

d. Event descriptions on the design and structure of the

AHRQ Common Formats for patient safety event reporting.

e. Existing aggregated, cross-tabbed and longitudinal data.

f. Previous analyses of data on trends in patient safety events and patterns of patient safety events.

g. Previous analyses of patient safety events included in the

National Healthcare Quality and Disparities Reports.

h. Background research conducted to help prepare for AHRQ report on effective strategies for reducing medical error and increasing patient safety.

i. Close-out documents from COR/PSO Team.

3.6 NPSD related content development and testing (within AHRQ maintained NPSD test and production environment): The Contractor shall begin work on data analysis and fine tuning the analysis to be incorporated in the table shells, reports, dashboard (refer to Tasks 12 and 13). The Contractor shall take approximately two (2) months to conduct data analytics, finalizing table shells, table contents (text report) along with discussions and determinations. Once the data/information (e.g., dashboards are finalized, the data/information are ready for public posting. The Contractor shall make sure ample time is allocated (e.g., for conducting data run/analysis, creating preliminary draft and finalizing data/reports, as well as accommodating iterative process (review, comments/feedback and revisions and clearance processes) for AHRQ staff as described in Task 8 and 12. After the finalized dashboards, and table contents, and supporting documents have been approved by AHRQ, the Contractor shall begin development of new dashboards and be ready to send over the updated dashboards and supporting documents to AHRQ Office of Communications for upload into the NPSD system. The contractor shall also follow with content testing processes once the designated dashboards are uploaded into the AHRQ maintained NPSD test and production environment.

4 Common Formats Maintenance and Upgrades:

Within HHS, AHRQ leads the development of the Common Formats for patient safety event reporting. The Contractor shall provide assistance and support with the development of the content for the Common Formats. The Common Formats can be viewed at www.psoppc.org.

Databases (NPSD) Systems Page 21 of 112

4.1 Maintenance of Current Common Formats: The Contractor shall maintain versioning and maintenance of the current Common Formats to include, but not limited to, supporting review of comments received and revision of the current version(s), any necessary technical edits to the reports and support materials (i.e., users guide, technical specifications – event descriptions, clinical notes, glossary, data dictionary, resource workbook, implementation guide, sample XML, flow chart), development of additional metadata attributes for the data elements, and revisions and upgrades to the technical specifications for the Common Formats. The Contractor shall review documents on an annual basis to make sure terminologies and definitions have not change over time (e.g., old terminologies have new meaning resulting in a change in the definitions/descriptions). All documents shall be 508 compliant. The Contractor shall review and update technical contents to the current Common Formats annually. The Contractor shall maintain a log for maintenance of Common Formats, available to the COR on a monthly basis.

4.2 Development of New Common Formats: In consultation with AHRQ, the Contractor shall develop new Common Formats reports and support materials (i.e., user guide, technical specifications –clinical notes, glossary, data dictionary, resource workbook, implementation guide, sample XML) for the revised or new content area. AHRQ will develop predecessor documents: event descriptions. The Contractor shall prepare these documents for pre-publication work meetings (e.g., AHRQ’s Patient Safety Work Group), final publication (as announced in the Federal Register), and posting to the National Quality Forum (NQF) Web site for stakeholder comments. Once comments are received from the National Quality Forum, and the NQF Expert Panel has prepared its feedback on the comments to AHRQ, the Contractor shall review, revise and develop any new content areas in consultation with AHRQ.

All documents shall be 508 compliant. A rough estimate of approximately one (1) Common Format is anticipated for development and one (1) for revision per year. AHRQ will make a determination which Common Format to prioritize and develop, which may include finalizing CFS-Hospital version 1.0.

4.3 Development of Metadata for the Common Formats and Transfer to

Metadata Registry: The Contractor shall maintain metadata for the Common Formats (both current and potential new content). The complete set of metadata for the Common Formats shall be maintained in a computerized system that allows entry, editing, and versioning of metadata attributes to provide document generation for and support development of AHRQ’s Common Formats. Working jointly with AHRQ, the Contractor shall determine which metadata attributes will be included in the content displayed at the United States Health Information Knowledgebase (USHIK). The Contractor shall coordinate metadata activities with the USHIK Contractor and transfer of metadata to USHIK with new releases of the Common Formats. Current information can be found at: https://ushik.ahrq.gov/mdr/portals. All

Databases (NPSD) Systems Page 22 of 112 documents shall be 508 compliant. The Contractor shall develop a project management timeline/milestone (e.g., IMS) and schedule work for developing, maintaining, and submitting/transferring metadata to Registry on an annual basis. The contractor shall obtain licenses for tools to perform this task. For example: Some tools used to perform this work are Visio, Excel, Sharepoint, and Tableau.

4.4 Health Level Seven (HL-7) Activities: The Contractor shall perform activities related to HL-7 and the Common Formats. Examples of such activities shall include preparation of documents/materials for ballot routing or documents related to the file transfer protocol, such as the current Implementation Guide. First draft available three (3) weeks after identifying needs. The Contractor shall finalize materials two (2) weeks after receiving comments from AHRQ. The Contractor shall be responsible for making sure the HL-7 version AHRQ is using meets industry standards and all Federal regulations (https://www.hhs.gov/hipaa/for-professionals/special-topics/hitech-act-enforcement-interim-final-rule/index.html). Current industry standards for the exchange, integration, sharing, and retrieval of electronic health information that supports clinical practice and the management, delivery and evaluation of health services is HL- 7.

However if current practice changes or evolves, the Contractor shall maintain and update current practice to reflect nuances in the Health Information Technology for Economic and Clinical Health (HITECH) (e.g., electronic medical records, meaningful use) industry. The Contractor shall conduct periodic assessment of all HL-7 activities in a log (e.g., monthly) and submit to the COR. The Contractor shall document changes and propose recommendations for changes to the COR. Final COR approval shall be obtained before changes are made.

4.5 Technical Specifications (TS) Support: The Contractor shall receive and track comments from AHRQ related to the Common Formats technical specifications. In addition, the Contractor shall maintain a service level agreement for the processing and tracking of technical specifications comments. The Contractor shall identify issues and log technical specifications. The Contractor shall periodically (e.g., monthly) assess technical specification documents and submit to the COR. After reviewing the documents, implement strategies to improve TS support.

4.5.1 Software Vendor Information and Support: Additional support activities to be developed for the Common Formats may include planning and coordination of conference calls (phone or video), in an effort to promote and provide detailed Common Formats technical specifications to the software developer and user audience. The Contractor shall plan on no more than four conference calls a year with the Common Formats patient safety reporting software community. The Contractor shall be responsible for working with AHRQ to develop content for the calls as well as logistics (e.g., phone/webinar information, materials) of the conference calls. The

Databases (NPSD) Systems Page 23 of 112

Contractor shall schedule call dates in advance. The Contractor shall send out conference materials (e.g., video/webinar links, call numbers, agendas, slides) twenty-four (24) hours prior to the conference call. The Contractor shall take notes and submit minutes (detailing the summary of the discussions, actions, next steps) in 3 days post conference call.

5. Common Formats Import Processor

5.1 Refine…