Attachment F - QASP (PSI).pdf

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

DEPARTMENT OF VETERANS AFFAIRS

Veterans Health Administration Office of Connected Care

Program Support Integration

Date: September 9, 2022

QASP Version Number: 0

QUALITY ASSURANCE SURVEILLANCE PLAN

For: Program Support Integration

Contract/Order Number: Blank until contract/order award

Contract/Order Description: OCC’s connected care technology network is an enterprise-wide, complex, and diverse system for the distribution, support, and sustainment of health care and related services. The scale of the program and its basis in unique, VA-developed products requires highly specialized industry support to successfully perform, implement and execute at the scope and complexity required by all program support integration functions. Program integration and project management support serve as the backbone of the product development and service delivery processes to ensure that the implementation of all VA-built products and services are tightly integrated and carefully coordinated with communications and training activities.

The Contractor shall provide integrated services and solutions in support of VA connected care technologies.

For the purposes of this contract, connected care is defined as the use of electronic information or telecommunications technologies to support health care delivery, patient and professional health-related education, public health, and health administration at a distance. “Connected care technologies” are defined as the combined VA ecosystem of clinical video equipment and software; store and forward technologies; web and mobile health applications; software-as-a-service (SaaS) / platform-as-a-service (PaaS) technologies; commercial-off-the-shelf (COTS) software products; mobile medical devices, peripherals, and wearables; remote patient monitoring technologies; online health portals; future connected care technologies; and ancillary items.

The Contractor shall provide support services in the following functional areas, as defined in PWS Section 5.2, which establishes all general requirements for the services performed under this contract. Specific requirements will be defined by VA at the individual Task Order (TO) level. To ensure integration across the program, the Contractor shall coordinate with all key stakeholders in the performance of orders placed against this contract.

Functional Areas:

a) Program Management and Program Integration Support (PWS 5.2.1)

b) Project, Release, and Implementation Management Support (PWS 5.2.2)

c) Communications Support (PWS 5.2.3)

d) Training Development, Delivery, and Implementation (PWS 5.2.4)

Contractor’s Name: Blank until contract/order award

1.0 PURPOSE

This Quality Assurance Surveillance Plan (QASP) provides a systematic method to monitor Contractor performance. This QASP describes:

• What will be monitored

• How monitoring will take place

• Who will conduct the monitoring?

• How monitoring efforts and results will be documented

This QASP does not detail how the contractor accomplishes the work. Rather, the QASP is created with the premise that the contractor is responsible for management and quality control actions to meet the terms of the contract. It is the Government’s responsibility to be objective, fair, and consistent in evaluating performance. In addition, the QASP should recognize that unforeseen and uncontrollable situations may occur.

This QASP is a “living document” and the Government may review and revise it on a regular basis. The Government can change the method of surveillance at any time however, will coordinate changes with the contractor. Updates shall ensure that the QASP remains a valid, useful, and enforceable document. Copies of the original QASP and revisions shall be provided to the contractor and Government officials implementing surveillance activities.

The following FAR clauses may apply depending on contract type:

37.6 Performance-Based Acquisition

46.4 Government Contract Quality Assurance

52.246-4 Inspection of Services – Fixed-Price

1.1 PERFORMANCE MANAGEMENT APPROACH

The PWS sets forth “what” service is required as well as the performance standards associated with that task, as opposed to “how” the Contractor should perform the work (i.e., results, not compliance). This QASP will define the performance management approach taken by the Office of Connected Care to monitor the Contractor’s performance to ensure the expected outcomes or performance standards communicated in the PWS are achieved. Performance management rests on developing a capability to review and analyze information generated through performance assessment. The ability to make decisions based on the analysis of performance data is the cornerstone of performance management; this analysis yields information that indicates to what extent the expected outcomes for the project are being achieved by the Contractor.

Performance management represents a significant shift from the more traditional quality assurance (QA) concepts in several ways. Performance management focuses on assessing whether outcomes are being achieved and to what extent. This approach migrates away from scrutiny of compliance with the processes and practices used to achieve the outcome. A performance-based approach enables the Contractor to play a large role in how the work is performed, as long as the proposed processes are within the stated constraints. Required processes are those required by law (federal, state, and local) and compelling business situations, such as safety and health. A “results” focus by the Government provides the Contractor flexibility to continuously improve and innovate over the course of the contract/order as long as the critical outcomes expected are being achieved and/or the desired performance levels are being met.

1.2 PERFORMANCE MANAGEMENT STRATEGY

The Contractor is responsible for the quality of all work performed. The Contractor measures that quality through the Contractor’s own quality control (QC) program. QC is work output, not workers, and therefore includes all work performed under this contract/order regardless of whether the work is performed by Contractor employees or by Subcontractors. The Contractor’s QC program will set forth the procedures for self-inspecting the quality, timeliness, responsiveness, customer satisfaction, and other performance requirements in the PWS.

The COR will conduct annual performance assessments in the CPARS system. The government representative(s) will monitor performance by the Contractor to determine how the Contractor is performing against performance standards. The Contractor will be responsible for making required changes in processes and practices to ensure performance is managed effectively. The Contractor will be monitored and assessed throughout the period of performance of the contract/order as to either meeting or not meeting the performance thresholds stated in the Performance Metrics Section of the Performance Work Statement (PWS). The Contracting Officer’s Representative (COR) will perform quarterly assessments.

The Performance Based Service Assessment, or other method, may be used to document this assessment. A Performance Based Service Assessment is provided at the end of this document. When Contractor performance is unacceptable, the COR will notify the Contractor Program Manager (CPM) and the Contracting Officer (CO).

Unacceptable performance is defined as; “the contractor is not meeting the Acceptable Levels of Performance (ALPs) as defined in the PWS or is in violation of any contract clause or terms and conditions. Notification of unacceptable performance issues shall be immediately provided to the CPM and shall not remain un-addressed until the end of an assessment period. In order to remediate performance issues in a timely manner, the COR should work collaboratively with the CPM. The COR/CO will engage the CPM to resolve the discrepancy.

1.3 QASP RELATION TO THE QUALITY CONTROL PLAN

The Contractor’s QCP is a formal contract deliverable. The Government expects the implementation of the Contractor’s QCP requirements will be sufficient in meeting the performance details of the PWS. While the QCP represents the way the Contractor will pursue quality and timeliness of services, as defined in the PWS, the QASP represents the way the Government will evaluate the Contractor’s performance. The Contractor’s QCP and the QASP should be complementary and ultimately ensure successful Contractor performance.

1.4 REVISIONS TO THE QASP

The QASP will be used in the Government’s administration of the contract and remains subject to revision at any time by the Government throughout the contract performance period. Revisions to this surveillance plan are the responsibility of the Contracting Officer (CO) or designee with support, input, and coordination with the VA program designees. An initial revision, encompassing surveillance details related to the Contractors submitted QCP may be made within 30 days of the initial QCP submission.

While the Contractor may be engaged in determinations of necessary revisions, changes may be made unilaterally at the discretion of the Government. As the performance period progresses, surveillance may be altered as determined necessary in-service areas where performance is either consistently excellent or consistently unsatisfactory. The Government may also alter surveillance as key priorities are determined by VA, OCC, or associated program management offices (PMO)s.

1.4.1 PERFORMANCE FEEDBACK

For instances where immediate notification of performance issues is not required, the COR should review the assessment in accordance with the quarterly reviews with the CPM and provide the assessment to CO. The COR/CO will notify the Contractor of the results for a rating of 3 or less, no later than 15 working days after the end of the assessment period.

2.0 GOVERNMENT ROLES AND RESPONSIBILITIES

The following personnel shall oversee and coordinate surveillance activities:

a. Contracting Officer (CO) - The CO shall ensure performance of all necessary actions for effective contracting, ensure compliance with the contract/order terms, and shall safeguard the interests of the United States in the contractual relationship. The CO shall also assure that the Contractor receives impartial, fair, and equitable treatment under this contract/order. The CO is ultimately responsible for the final determination of the level of acceptability of the Contractor’s performance.

Assigned CO: David Barber, Contracting Officer

Organization: Department of Veterans Affairs, Office of Procurement, Acquisition and Logistics, Strategic Acquisition Center

b. Contracting Officer’s Representative (COR) - The COR is responsible for technical administration of the contract/order and shall assure proper Government surveillance of the Contractor’s performance. The COR shall revise the QASP to meet the specifics of the submitted QCP and also keep a quality assurance file including (but not limited to) quality assessment reports, quarterly quality review details and annual CPARS. At the conclusion of the contract or when requested by the CO, the COR shall provide documentation to the CO. The COR is not empowered to make any contractual commitments or to authorize any contractual changes on the Government’s behalf.

Assigned COR: TBD

c. Other Key Government Personnel – TBD

3.0 CONTRACTOR REPRESENTATIVES

a. Program Manager – To be completed at award

b. Other Contractor Personnel – To be completed at award; if any (name and title)

4.0 PERFORMANCE STANDARDS

Performance Standards define required performance for each of the performance objectives. The Government performs surveillance to determine if the Contractor exceeds, meets or does not meet these standards.

The Performance Metrics for Performance Standards are defined in Section 6.5 of the PWS and Section 10 of the QASP. The Government may utilize the Performance Based Service Assessment, provided at the end of this document, or other methods to compare Contractor performance to the ALPs.

5.0 METHODS OF QA SURVEILLANCE

Various methods exist to monitor performance. The COR shall use any or a combination of the surveillance methods listed below in assessing performance using this QASP.

1. Inspection of a Random Sample of deliverables and Contractor performance for all PWS Tasks.

a. Each quarter, the COR shall review a random sampling of the Contractor’s performance and generated documentation. The results of this review shall be documented and placed in the COR’s QA file.

2. Validated Stakeholder Complaint or Validated Below Average Acceptable Level of Performance (ALP) in a Specific Area

a. Each quarter, the COR shall review the Contractor’s performance and/or generated documentation corresponding to a validated stakeholder complaint or validated inability to perform in accordance with the ALP in a specific area. The results of this review shall be documented and placed in the COR’s QA file.

3. Performance Management Reviews

a. At least each quarter, the COR shall hold a Performance Management

Review (PMR) meeting with the Contractor. The COR shall review findings from the PMR and document the results accordingly. This assessment shall be placed in the COR’s QA file.

4. Direct Observation

a. Can be performed periodically or through 100% surveillance by the COR and or designates.

5. Periodic Inspection.

a. Uses a comprehensive evaluation of selected outputs. Inspections may be scheduled [Daily, Weekly, Monthly, Quarterly, or annually] or unscheduled, as required.)

6. Progress or Status Meetings.

a. Status Meetings; at a minimum Monthly review and daily status briefing/meetings with key contract personnel, support personnel, and onsite federal representation. Progress Meetings; meetings directed by the Government and contractor that can include but are not limited to, the execution of a separate Task Order.

Surveillance results may be used as the basis for revisions to Deliverables and to substantiate CPARS review ratings. In such cases, the Inspection of Services clause in the Contract becomes the basis for the CO’s actions.

6.0 ACCEPTABLE LEVELS OF PERFORMANCE (ALP)

Metrics and methods are designed to determine if performance exceeds, meets, or does not meet a given standard and ALP.

The ALPs are included in the Performance Metrics Section of the PWS and Section 10 of the QASP for Contractor performance and are structured to allow the Contractor to manage how the work is performed, while providing negative incentives for performance shortfalls.

7.0 INCENTIVES

The Government shall consider the Contractor’s performance when making a determination to exercise any options.

8.0 DOCUMENTING PERFORMANCE

a. ACCEPTABLE PERFORMANCE

The Government shall document acceptable performance accordingly. Any report may become a part of the supporting documentation for any contractual action.

b. UNACCEPTABLE PERFORMANCE

When unacceptable performance occurs, the COR shall inform the CO. This will always be in writing although when circumstances necessitate immediate verbal communication, that communication will be followed up in writing. The COR shall document the discussion and place it in the COR file.

When the CO determines formal written communication is required, the COR shall prepare a Contract Discrepancy Report (CDR), and present it to the Contractor's program manager.

The Contractor shall acknowledge receipt of the CDR in writing to the CO. The CDR will state how long after receipt the Contractor has to take corrective action.

The CDR will also specify if the Contractor is required to prepare a corrective action plan to document how the Contractor shall correct the unacceptable performance and avoid a recurrence. The CO shall review the Contractor's corrective action plan to determine acceptability.

Any CDRs may become a part of the supporting documentation for any contractual action deemed necessary by the CO.

9.0 FREQUENCY OF MEASUREMENT

a. Frequency of Measurement.

During contract/order performance, the COR will periodically analyze whether the negotiated frequency of surveillance is appropriate for the work being performed, and at a minimum shall be quarterly.

b. Frequency of Performance Assessment Meetings.

The COR shall meet with the Contractor quarterly to assess performance and shall provide a written assessment to the CO.

10.0 PERFORMANCE METRICS

Performance Objective

Performance Standard Acceptable Levels of Performance

A. Technical / Quality of Product or Service

1. Demonstrates understanding of requirements

2. Efficient and effective in meeting requirements

3. Meets technical needs and mission requirements

4. Provides quality services/products

Satisfactory or higher

B. Project Milestones and Schedule

1. Established milestones and project dates are met

2. Products completed, reviewed, delivered in accordance with the established schedule

3. Notifies customer in advance of potential problems

Satisfactory or higher

C. Cost & Staffing 1. Currency of expertise and staffing levels appropriate

2. Personnel possess necessary knowledge, skills and abilities to perform tasks

Satisfactory or higher

D. Management 1. Integration and coordination of all activities to execute effort

2. Responsive to and cooperative with all stakeholders and business relations, including VA personnel, VA designees, and other Contractors working for or on behalf of OCC

Satisfactory or higher

Quality of submission should also be considered. See examples below. Error rates or resubmits for content flaws would be the measures associated with these standards.

• Accuracy - Work Products shall be accurate in presentation, technical content, and adhere to accepted elements of style.

• Clarity - Work Products shall be clear and concise. Any/All diagrams shall be easy to understand and be relevant to the supporting narrative.

• Consistency to Requirements - All work products must satisfy the requirements of this PWS.

• File Editing - All text and diagrammatic files shall be editable by the VA in Windows-based or Adobe environments/platforms.

• Format - Follow specified VA Directives or Manuals and/or best business practices.

• Presentations - Presentations shall be clear, concise, executive-focused, and written in plain, clear English with minimal jargon, understandable by lay persons.

The quality of deliverables directly contributes to organizational communications.

• Project Plan - Project Plan shall be comprehensive; recognize and address authority, perceptions, and concerns of stakeholders; incorporate scope of requisite requirements across the organization and/or agency.

• Reports - There shall be no omissions in the reports, documents or functional requirements.

• Publications and other documents - Deliverables shall be in formats appropriate to target audiences; user friendly, clear, thorough and comprehensive.

• Meeting support - Pre-meeting preparations and logistics demonstrate smooth meeting operations; complete comprehensive post-meeting summaries to include but not limited to: Minutes, Action Items, Attendees, Program Objectives and Milestones and major decision points.

• Analyses and Assessments - Analyses and assessments are performed with accuracy, completeness and adherence to industry best practices.

Obtain stakeholder input. Deliverables shall consist of the timely implementation of input mechanisms, and shall consist of an accurate and comprehensive synthesis of results and recommendations. Integration of relevant stakeholder input documented for each deliverable.

EVALUATION RATINGS DEFINITIONS

Rating Definition Notes

Exceptional

Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

To justify an Exceptional rating, identify multiple significant events and state how they were of benefit to the Government. A singular benefit, however, could be of such magnitude that it alone constitutes an Exceptional rating. Also, there should have been NO significant weaknesses identified.

Very Good

Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

To justify a Very Good rating, identify a significant event and state how it was a benefit to the Government. There should have been no significant weaknesses identified.

Satisfactory Performance meets contractual requirements. The contractual performance of the element or sub-element contains some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

To justify a Satisfactory rating, there should have been only minor problems, or major problems the contractor recovered from without impact to the contract/order. There should have been NO significant weaknesses identified. A fundamental principle of assigning ratings is that contractors will not be evaluated with a rating lower than Satisfactory solely for not performing beyond the requirements of the contract/order.

Marginal Performance does not meet some contractual requirements. The contractual performance of the element or sub-element being evaluated reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented.

To justify Marginal performance, identify a significant event in each category that the contractor had trouble overcoming and state how it impacted the Government. A Marginal rating should be supported by referencing the management tool that notified the contractor of the contractual deficiency (e.g., management, quality, safety, or environmental deficiency report or letter).

Unsatisfactory Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub-element contains a serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

To justify an Unsatisfactory rating, identify multiple significant events in each category that the contractor had trouble overcoming and state how it impacted the Government. A singular problem, however, could be of such serious magnitude that it alone constitutes an unsatisfactory rating. An Unsatisfactory rating should be supported by referencing the management tools used to notify the contractor of the contractual deficiencies (e.g., management, quality, safety, or environmental deficiency reports, or letters).

PERFORMANCE BASED SERVICE ASSESSMENT

CONTRACTOR: ________________________

GOVERNMENT REQUIRING ACTIVITY: _____________________

CONTRACT/ORDER NUMBER/ TITLE: _____________________

PERFORMANCE PERIOD COVERED: _____________________________________

NAME AND TITLE OF COR: _______________________________

DATE: __________________________

EVALUATION RATINGS FOR ASSESSMENT

EXCEPTIONAL

VERY GOOD

SATISFACTORY

MARGINAL

UNSATISFACTORY

All value ratings must be supported, objective and explained in the Narrative Section for each Performance Objective.

PERFORMANCE OBECTIVES:

A. TECHNICAL/QUALITY OF PRODUCT OR SERVICE: Rating: <Value>

How well does the contractor meet your Technical Requirement IAW the performance metrics in the PWS?

NARRATIVE: (enter narrative in box)

B. PROJECT MILESTONES AND SCHEDULE: Rating: <Value>

How well does the contractor meet the established schedule IAW the performance metrics in the PWS?

C. COST & STAFFING: Rating: <Value>

Are the staffing levels and expertise appropriate for accomplishing the mission IAW the performance metrics in the PWS?

Were the invoices current, accurate and complete?

D. MANAGEMENT: Rating: <Value>

How well did the contractor integrate/coordinate all activities needed to execute the contract IAW the performance metrics in the PWS?