L-5 Quality Assurance Surveillance Plan (QASP).docx

CUI

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

1. Contract Title Joint Warfighting Cloud Capability (JWCC) Unified Cloud Marketplace (UCM) Core

2. Purpose:

This QASP provides a systematic method to evaluate performance for the stated contract. This QASP explains the following:

· What will be monitored?

· How monitoring will take place.

· Who will conduct the monitoring?

· How monitoring efforts and results will be documented.

This QASP does not detail how the contractor accomplishes the work. Rather, the QASP is created with the premise that the contractor is responsible for management and quality control actions to meet the terms of the contract. It is the Government’s responsibility to be objective, fair, and consistent in evaluating performance. In addition, the QASP should recognize that unforeseen and uncontrollable situations may occur.

This QASP is a “living document” and the Government may review and revise it on a regular basis. Updates shall ensure that the QASP remains a valid, useful, and enforceable document. Copies of the original QASP and revisions shall be provided to the contractor and Government officials implementing surveillance activities.

The purpose of this plan is to identify the methods and procedures the Government will use to ensure it receives the offerings required under this contract, and the associated task orders (TOs), including those measures and metrics identified in the Performance Work Statement (PWS).

2.1. Intent.

The intent of this plan is to allow the Government to consistently and fairly monitor the Contractor’s quality control practices applied to delivered outcomes. In the event there is a quality issue, this plan provides the basis to identify any weaknesses or deficiencies and enables the Government to provide the Contractor with the necessary information required for corrective actions to be taken to improve quality, processes and procedures. The Contractor is responsible for developing an effective and efficient methodology to ensure they meet and/or exceed the required thresholds of service stipulated in this contract. The Government intends to perform surveillance on this contract in accordance with this Quality Assurance Surveillance Plan (QASP) but reserves the right to monitor the contract in any manner and at any time necessary to ensure that the services provided conform to contract requirements. The Government reserves the right to apply quality assurance at the subcontract level, and may perform quality assurance at the Contractor's, or subcontractor’s, place of business. Non-conforming services discovered with subcontractors will be identified to the prime Contractor for resolution.

2.2. Authority.

Authority for issuance of this QASP is provided under Part 46 of the Federal Acquisition Regulation (FAR), Inspection of Services clauses, which provide for inspection, acceptance and documentation of the services called for in the contract or orders. This acceptance is to be executed by the Contracting Officer (KO) or a duly authorized representative.

3. Roles and Responsibilities:

The following personnel shall oversee and coordinate surveillance activities.

Program/Project Manager (PM) – The PM provides program oversight and supports the COR’s performance assessment activities.

Contracting Officer (KO) – The KO shall ensure performance of all necessary actions for effective contracting, ensure compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The KO shall also ensure that the contractor receives impartial, fair, and equitable treatment under this contract. Determine the final assessment of the contractor’s performance.

Contracting Officer’s Representative (COR) – The COR is responsible for providing continuous technical oversight of the contractor’s performance. The COR uses the QASP to conduct the oversight/surveillance process. The COR shall keep a Quality Assurance file that accurately documents the contractor’s actual performance. The purpose is to ensure that the contractor meets the performance standards contained in the contract. The COR is responsible for reporting early identification of performance problems to the KO. The COR is required to provide an annual performance assessment to the KO which will be used in documenting past performance. The QASP is the primary tool for documenting contractor performance. The COR is not empowered to make any contractual commitments or to authorize any contractual change on the Government’s behalf.

Other Key Government Personnel – This may include performance monitors, inspectors, technical experts, workforce service division and center directorate level security personnel, requirements office, or others (i.e. users) who provide information that helps the COR monitor contractor performance.

Contractor – Roles and Responsibilities of the Contractor as related to quality assurance are as follows:

· Complies fully with the terms and conditions of the contract.

· Participates as a member of the multi-functional team after contract award.

· Maintains and implements a quality control plan that is acceptable to the Government.

· Ensures that non-conformance of contract requirements is identified, corrected, and prevents recurrence.

· Tenders to the Government for acceptance only those services that conform to contract requirements.

· Recommends any changes to the contract that will provide more effective operations or eliminate unnecessary costs

4. Primary Method of Surveillance:

A combination of random sampling, 100% inspection and stakeholder input will be used to evaluate contractor performance for this IDIQ. Periodic Inspection applies to all required deliverables and Contract Data Requirement Lists (CDRLs), stakeholder input may be utilized to either initiate inspections or supplement any issues discovered resulting from these inspections.

The performance standards, acceptable quality level (AQL) and surveillance methods are outlined in Section 5. In addition to the Performance Standards defined in Section 5 below, the Government shall perform annual surveillance for contractor compliance as required by FAR 52.222-50, Combating Trafficking in Persons (see Section 7 below).

· 100 Percent Inspection. This is usually only the most appropriate method for infrequent tasks or tasks with stringent performance requirements, e.g., where safety or health is a concern. With this method, performance is inspected/evaluated at each occurrence. One hundred percent inspection is too expensive to be used in most cases.

· Random Sampling. This is usually the most appropriate method for recurring tasks. With random sampling, services are sampled to determine if the level of performance is acceptable. Random sampling works best when the number of instances of the services being performed is very large and a statistically valid sample can be obtained. Computer programs may be available to assist in establishing sampling procedures.

· Periodic Inspection: These services are monitored weekly, monthly, quarterly, semiannually, annually, etc. Periodic types of activities are perfect for periodic inspection because not only are they infrequent, but there is normally a predetermined, specified time frame within which the tasks must be accomplished. Therefore, you know exactly when to conduct the evaluations. Periodic inspections automatically become 100 percent evaluations or "100 percent checks." Inspections should be divided and scheduled by frequency: annual, semiannual, quarterly, monthly, weekly and as required. Sometimes services are required for which the time or frequency cannot be predicted, such as accident investigations, one-time special tasking by higher headquarters, etc. These would be labeled "as required inspections." Others are known and predictable such as the quarterly status report or the monthly travel report currently included in some DISA service contracts.

· Stakeholder Input. Although usually not a primary method, this is a valuable supplement to more systematic methods. For example, in a case where random sampling indicates unsatisfactory service, customer complaints can be used as substantiating evidence. In certain situations where customers can be relied upon to complain consistently when the quality of performance is poor, e.g., dining facilities, building services, customer surveys and customer complaints may be a primary surveillance method, and customer satisfaction an appropriate performance standard. In all cases, complaints should be documented, preferably on a standard form.

Surveillance Matrix

The Surveillance Matrix is the list of performance objectives and standards that must be performed by the contractor. This matrix details the method of surveillance the COR will use to validate and inspect these performance elements. Inspection of each element will be documented in the COR file.

Performance objectives define the desired outcomes. Performance Standards define the level of service required under the contract to successfully meet the performance objective. The inspection methodology defines how, when, and what will be assessed in measuring performance. The Government performance surveillance, using this QASP, to determine the quality of the contractor’s performance as it relates to the performance element standards. The AQL performance Standard Table should be used to form the foundation of the COR’s inspection checklist.

5. Performance Standards:

6.1.1 Identify Key Personnel

6.1.2.2.1 Program Management Plan

6.1.2.2.1.4 Ad Hoc Reporting/Information Products

6.1.2.2.2 Communication Plan

6.1.2.2.3 Risk Management Plan

6.1.2.2.4 Quality Control Plan

6.1.2.2.5 Information Technology Service Management (ITSM) Plan

6.1.2.2.6 Lifecycle Management Plan

6.2.4.3 Ordering Guide Inputs

6.2.10 Training and Documentation

6.3.1 Cybersecurity Management Plan

6.3.4 Support of DoD Zero Trust Strategy

6.3.6.1 Controlled Cryptographic Item Management Plan

6.3.9.1 Supply Chain Risk Management Plan

6.3.12.2 Self-Certification of Control

6.3.12.2.2 Co-Use Agreements

6.3.13 System and Organization Controls (SOC) 1 Type II

6.6.2 Policy Implementation, Automation, and Monitoring

Contract Management (CDRL A001)

6.1.2 Presentation Materials

Requirements per CDRL A002 Kickoff Meeting Materials required two business days prior to Kickoff IPR Meeting Materials required three business days prior to IPR TEMs and all other meeting materials within timelines agreed to, upon Government request Minimum Acceptable:

90% on-time delivery of Meeting Materials 100% inspection

6.1.2.2.1.4 Ad Hoc Reporting/Information Products

Requirements per CDRL A008

6.1.2.2.8 Contract Monthly Progress Report (CMPR)

Requirements per CDRL A003

6.1.3 JWCC UCM Core Service Catalog

Requirements per CDRL A004

6.2.2.3.2 Exception requests for “high availability” capability

6.2.2.7.2 Data Center Timeline Extension Request

6.2.2.2 Exception requests for Contractor Lifecycle Management Plan

6.2.8.1.1 Exception requests for Artificial Intelligence and Advanced Data Analytics

6.2.8.1.2 Exception requests for Artificial Intelligence and Advanced Data Analytics

6.2.11.8 Exception Request for JWCC Console Third-Party Service Provider (3SP) Mandatory Addendum to Licens Agreements or SLAs

6.4.3.3 Extension Request for DoD Account Provisioning API

Requirements per CDRL A006, JWCC UCM Core Exception/Extension Request PWS requirements 90% of the time.

6.2.4.2 Contractor Cloud Information Portal

6.2.2.5 – Service Incident Reports Service Incident Follow-up Reports Service Incident After-Action Reports

Requirements per CDRL A005

6.2.2.6. - Service Health Dashboard

6.2.3.4 - Peering Report Requirements per CDRL A08

6.2.4.4 - Contractor Calculator

6.2.6.2 - Special Access Program (SAP) Requirements per CDRL A007

6.2.6.3 - Cross-Domain Solution Requirements per CDRL A007

6.2.7 - Edge Device Specifications and Hardware Asset Management Processes Requirements per CDRL A009 The Contractor shall meet the mutually agreed-to requirements 90% of the time

100% inspection

6.2.11 - JWCC UCM Core Console Deliverable

6.3.5.3 - Security Authorization Letters

6.3.5.5 - SAP Authorization Information

6.3.5.6 - CDS Authorization Information Requirements per CDRL A007 The Contractor shall meet the mutually agreed-to requirements 90% of the time Access to authorization BoE within five days of Government request 90% of the time 100% inspection

6.3.2 Auditing, Logging and Monitoring

Requirements per CDRL A08 The Contractor shall meet the mutually agreed-to requirements 90% of the time

Periodic inspection, as required

6.3.6.1 - CCI Tracking Documents (SF-153, Orders, Returns, Transfers, Needs) Requirements per CDRL A007

6.3.7 – Vulnerability, Cybersecurity Incident, and Breach Capability Deliverable

6.3.10.2 - Cybersecurity Incident Reports Requirements per CDRL A005

6.3.10.3 Cyber Cooperation and Information Sharing

6.3.10.4 Cybersecurity Queries

Requirements per CDRL A08

6.3.10.9 Execution of Directives

6.3.10.8 Third Party Marketplace Vulnerability Report

Requirements per CDRL A005

6.4.1 - Secure Data Transfer Deliverable

6.4.2 - Contractor API and End Point Link Deliverable

6.4.3 - DoD Account Provisioning Tool API Deliverable

6.6.1.1 - FinOps Data set Deliverable

6.6.1.1 - FinOps Documentation Requirements per CDRL A001

6.6.1.4 - Spend Threshold Capability Deliverable

6.6.1.5 - Billing Capability (Time-based & Real-Time) Deliverable

6.6.1.6 - Financial Reporting Capability Deliverable

6.6.2 - Policy Implementation Capability Deliverable

6.7 - Desired Capabilities and Emerging Technologies Requirements per CDRL A002

12.1 Government-Furnished Property Reporting

Requirements per CDRL A008

6. Documenting Performance:

Acceptable Performance.

The Government shall document positive performance. A Performance Assessment Report (PAR) template is attached (Appendix 1). Any report may become part of the supporting documentation for fixed price payments, award/incentive fee payments, Contractor Performance Assessment Reporting System (CPARS) entries, or other actions. Upon Government acceptance of the corrected services the contractor will be permitted to invoice for the 0.5% administrative billing fee associated with that specific task order.

Unacceptable performance.

Unacceptable contractor performance can be identified by customer complaints (Appendix 2) or upon schedule surveillance evaluations. When unacceptable performance occurs, the COR shall inform the contractor. This will normally be in writing unless circumstances necessitate verbal communication and may be through the PAR template (Appendix 2). The COR shall document the discussion and place it in the COR file and provide a copy to the KO.

When the COR determines formal written communication is required, the COR shall prepare a Corrective Action Report (CAR). The CAR is forwarded to the KO for dissemination to the contractor. A CAR template is attached to this QASP (Appendix 3).

The contractor shall acknowledge receipt of the CAR in writing. The CAR will specify if the contractor is required to prepare a corrective action plan to document how the contractor shall correct the unacceptable performance and avoid a recurrence. The CAR will also state how long after receipt the contractor has to present this corrective action plan to the KO and COR. The Government may withhold payment for services until corrective action is taken. The Government shall review the contractor's corrective action plan to determine acceptability. Upon Government acceptance of the corrected services the contractor will be permitted to invoice for the service fee.

Any CAR may become a part of the supporting documentation for contract payment deductions, fixed fee deductions, award fee nonpayment, CPARS or other actions deemed necessary by the KO.

7. Mandatory or regulatory compliance items:

The contractor will provide initial proof of compliance with regulatory and compliance items contained in the contract. The Contracting Officer's Representative (COR) and the Contracting Officer (KO) will evaluate the initial submission and will conduct ad hoc reviews of the contractor's compliance throughout the remainder of the contract. The contractor will monitor compliance as required by FAR 52.222-50, Combating Trafficking in Persons, and report any issues to the KO. The contractor shall inform its employees in writing of contractor employee whistleblower rights and protections as required by DFARS 252.203-7002, Requirement to Inform Employees of Whistleblower Rights. The contractor shall also ensure that the clause is included in all subcontracts. No later than 60 days prior to expiration of the period of performance, the contractor shall provide written affirmation to the COR that a written notification was provided to Contractor employees and the forum/means in which the written notification was provided (e.g., posted in hallways, cubicle spaces, common room spaces, email correspondence).

8. Acceptable Quality Level (AQL):

AQL for tasks on this IDIQ are detailed in Section 5 above.

9. Evaluation Method:

The COR will distribute deliverables to the Program Management Office for technical review and recommendations for acceptance. The COR will review all contractor deliverables for completeness, accuracy, and compliance with the requirements outlined in the contract. Furthermore, the COR will continuously monitor contractor performance against the Performance Requirements through observation, sampling, and review of contractor-provided performance data. Any identified deficiencies in deliverables or performance will be documented and communicated to the contractor in writing, including specific corrective actions required and a specific timeline for completion. Repeated or significant performance failures will be escalated according to the procedures established in the contract and the separate Evaluation Plan.

10. Performance Rating

In evaluating the quality of contractor’s performance, the following performance ratings may be used (same as CPAR ratings).

Appendix 1 – Performance Assessment Report

PERFORMANCE ASSESSMENT REPORT (PAR)

(If more space is needed, use reverse and identify by number)

4. QUALITY ASSURANCE PERSONNEL (COR) SIGNATURE AND DATE

I. PERFORMANCE

7. |_| DEFICIENCY (CHECK ALL BOXES THAT APPLY)

|_| NEW

|_| REPEAT

|_| NO DEFICIENCY NOTED

8. SERVICES SUMMARY or PWS PARAGRAPH ITEM REVIEWED

II. CONTRACTOR VALIDATION

11. CONTRACTOR REPRESENTATIVE |_| CONCUR |_| NON-CONCUR

12. CORRECTIVE ACTION ESTIMATED COMPLETION DATE

13. CONTRACTOR REPRESENTATIVE CORRECTIVE ACTION AND PREVENTION OF RECURRENCE OR REASON FOR NON-CONCURRENCE OF COR CITED DEFICIENCY

III. ACTION CORRECTED

14. |_| CONCUR |_| NON-CONCUR COR SIGNATURE AND DATE

15. COR REMARKS (REQUIRED)

6. CONTRACTOR REPRESENTATIVE REMARKS

Appendix 2 – Customer Complaint Record

CUSTOMER COMPLAINT RECORD

DATE/TIME OF COMPLAINT

SOURCE OF COMPLAINT

ORGANIZATION

BUILDING NUMBER

INDIVIDUAL

PHONE NUMBER

NATURE OF COMPLAINT

CONTRACT REFERENCE

VALIDATION

DATE/TIME CONTRACTOR INFORMED OF COMPLAINT

ACTION TAKEN BY CONTRACTOR

RECEIVED/VALIDATED BY

Appendix 3 – Corrective Action Report

CORRECTIVE ACTION REPORT (CAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACTOR

2. CONTRACT NUMBER

3. TYPE OF SERVICES

4. FUNCTIONAL AREA

5. SUSPENSE DATE

6. CONTROL NUMBER

7. DEFICIENCY |_| MAJOR |_| MINOR

FINDING:

FINDING IMPACT:

Please respond with a written corrective action plan that details the corrective action of the cited deficiency, the cause of the deficiency, and actions taken to prevent recurrence by Suspense Date in Block 5. If date was not entered in Block 5, the contractor is not required to provide a response.

8. QUALITY ASSURANCE PERSONNEL (COR)

TYPED NAME AND GRADE

SIGNATURE AND DATE

9. ISSUING AUTHORITY

TYPED NAME AND GRADE

SIGNATURE AND DATE

10. COR RESPONSE TO CONTRACTOR CORRECTIVE ACTION AND ACTION TAKEN TO PREVENT RECURRENCE

11. COR DETERMINATION

|_| ACCEPTED |_| REJECTED

12. CLOSE DATE

Controlled By: DISA Controlled By: J-924 CUI Category: PROCURE Distribution/Dissemination Control: FEDCON POC: disa.meade.j-9.list.jwcc-next@mail.mil