A01 - Attachment 0005 - Quality Assurance Surveillance Plan_DCGS-A CD 2_....pdf

UNCLASSIFIED // FOR OFFICIAL USE ONLY

Quality Assurance Surveillance Plan (QASP)

FOR

Distributed Common Ground System – Army (DCGS-A)

Capability Drop (CD) 2 – Data Fabric and Analytics

Solicitation Number: W56KGY-19-R-0007

Version 1.0

26 June 2019

Project Manager (PM) DCGS-A 6580 Surveillance Loop, Building 6006 Aberdeen Proving Ground, MD 21005

UNCLASSIFIED // FOR OFFICIAL USE ONLY

1. Introduction

This QASP is pursuant to the requirements listed in the Statement of Work (SOW) for the DCGS-A CD 2 requirement. This QASP sets forth the procedures and guidelines that the Government will use in evaluating the performance of the contractor.

2. Mission

This Multiple Award indefinite delivery, indefinite quantity contract has a seven year ordering period (4 year base and 3 year option). Those orders issued could be for the following activities: Software Delivery; Intelligence Analyst Testing; Technical Testing;

Operational Testing; Joint and Army Interoperability Certifications; NIPR, SIPR and JWICS Accreditation; contractor Field Service Support; and Training. Individual DOs will be executed for the requirements as specified in the SOW.

3. Purpose

The role of the Government in Quality Assurance (QA) is to ensure contract performance standards are achieved. This QASP is a Government-developed document used to determine if the contractor’s performance meets the performance standards in the contract. The QASP establishes procedures and measures on how this assessment/inspection process will be conducted, and provides details for continuous oversight process including:

• What will be monitored

• How monitoring will take place

• Who will conduct the monitoring

• How monitoring efforts and results will be documented

The contractor is responsible for implementing performance and delivering contract items that meets contract standards using its own organizational quality plans. The QASP provides the structure for the government’s surveillance of the contractor’s performance to assure that it meets contract performance requirements. It is the government’s responsibility to be objective, fair and consistent in evaluating contractor performance.

The QASP is not part of the contract nor is it intended to duplicate the contractor’s quality control plan. This QASP is a living document. Flexibility in the QASP is required to allow for an increase or decrease in the level of surveillance necessary based on contractor performance.

The Government may provide a copy of the QASP to the contractor to facilitate open communication. In addition, the QASP should recognize that unforeseen or uncontrollable circumstances might occur that are outside the control of the contractor and how those potential situations will be managed.

Bottom line, the QASP should ensure early identification and resolution of performance issues to minimize impact on mission performance.

4. Authority

Authority for issuance of this QASP is provided under the Federal Acquisition Regulation Part 46, Quality Assurance and Inspection of Services clauses, which provides for inspection, acceptance and documentation of the service called for in the contract or order. This acceptance is to be executed by the contracting officer or designated Contracting Officer’s Representative (COR).

5. Roles and Responsibilities

The following personnel shall oversee and coordinate surveillance activities:

Procurement Contracting Officer (PCO) – The PCO shall ensure performance of all necessary actions for effective contracting, ensure compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The PCO shall also ensure that the contractor receives impartial, fair, and equitable treatment under this contract. Determine the final assessment of the contractor’s performance.

Contracting Officer’s Representative (COR) - The COR is responsible for providing continuous technical oversight of the contractor’s performance. The COR uses the QASP to conduct the oversight/surveillance process. The COR shall keep a Quality Assurance file that accurately documents the contractor’s actual performance. The purpose is to ensure that the contractor meets the performance standards contained in the contract.

The COR is responsible for reporting early identification of performance problems to the PCO. The COR is required to provide an annual performance assessment to the PCO which will be used in documenting past performance. The QASP is the primary tool for documenting contractor performance. The COR is not empowered to make any contractual commitments or to authorize any contractual change on the Government’s behalf. Any contractual change is solely the PCO's responsibility.

Project/Product Manager (PM) – The PM provides primary program oversight, nominates the COR, ensures the COR is trained before performing any COR duties and supports the COR’s performance assessment activities. While the PM may serve as a direct conduit to provide Government guidance and feedback to the Contractor on technical matters, they are not empowered to make any contractual commitments or any contract changes on the government’s behalf.

PM DCGS-A Program Analyst - PM DCGS-A Program Analyst will be an individual designated by PM DCGS-A to act as the authorized representative to assist in the financial management of the contract and subsequent orders.

6. Performance Requirements and Method of Surveillance

6.1. Contract Surveillance

The goal of the QASP is to ensure that contractor performance is effectively monitored, documented, and altered as necessary. The COR’s contribution is their professional relationships with the PCO, PM and the contractor, which enables positive, open and timely communications. The foundation of this relationship is built upon objective, fair, and consistent COR evaluations of contractor performance against contract requirements.

The COR will use methods, such as in-person/on-site surveillance, review of data deliverables, user feedback, and others, to ensure the contractor complies with all contract requirements. Specific methods and frequency will be specific at the order level;

however, at a minimum will be conducted on a monthly-basis. The COR is responsible for a wide range of surveillance requirements that effectively measure and evaluate the contractor’s performance. Additionally, this QASP is based on the premise that the contractor, not the government, is responsible for management and quality control actions to successfully meet the terms of the contract.

6.2. Surveillance Metrics

The Surveillance Metrics (in the table below) is the list of performance objectives and standards that must be performed by the contractor. This table details the method of surveillance the COR will use to validate and inspect these performance elements.

Inspection of each element will be documented in the COR file.

Performance objectives define the desired performance outcomes for the requirements they represent. Performance standards define the level of acceptable service required under the contract to meet the performance objective. The inspection methodology defines how, when, and what will be assessed in measuring performance. The Government performs surveillance, using this QASP, to determine the quality of the contractor’s performance as it relates to the performance element standards.

The following table is the foundation of the template for the COR to perform surveillance on each of the delivery orders as defined in section 3 of the SOW. The contractor will propose a quality assurance effort for each of the delivery order proposals. The COR will use the table below to measure the performance against the surveillance metrics using the SOW requirement.

DCGS-A Capability Drop 2 (CD 2) - QASP

UNCLASSIFIED // FOR OFFICIAL USE ONLY

Elements (SOW Paras)

Quality Assurance Surveillance Plan

(QASP)

Surveillance Method

Frequency Incentives Positive/ Negative

Unsatisfactory (Red)

Satisfactory (Green)

Excellent (Dark Blue)

3.1 Intelligence Analyst Testing

3.2 Technical

Testing

The Contractor failed to deliver work products or provide support in accordance with the SOW below 90% on time with more than 10% of the product or support requiring rework.

The Contractor delivered work products or provided support in accordance with the SOW 90-94% on time with 6-10% of the product or support requiring rework.

The Contractor exceeded the delivered work product requirements or provided support in accordance with the SOW.

Product was delivered at least 95% on time with no more than 5% of the product or support requiring rework.

Customer and Government subject matter experts, Monthly feedback

Per each test event

The results of this evaluation will be reflected in CPARs/PPIRS.

Favorable consideration of One Year Option Year if score is greater than 90 with no score less than satisfactory.

3.3 NIPRNet

Accreditation

3.4. SIPRNet

3.5 JWICS

The Contractor failed to deliver work products or provide support in accordance with the SOW below 90% on time with more than 10% of the product or support requiring rework.

The Contractor delivered work products or provided support in accordance with the SOW 90-94% on time with 6-10% of the product or support requiring rework.

The Contractor exceeded the delivered work product requirements or provided support in accordance with the SOW.

Product was delivered at least 95% on time with no more than 5% of the product or support requiring rework.

Customer and Government subject matter experts, Monthly feedback

Monthly The results of this evaluation will be reflected in CPARs/PPIRS.

Favorable consideration of One Year Option Year if score is greater than 90 with no score less than satisfactory.

3.7 Army

Interoperability Certification

3.8 Joint

Interoperability Certification

3.10 Operational

Assessment Phase

The Contractor failed to deliver work products or provide support in accordance with the SOW below 90% on time with more than 10% of the product or support requiring rework.

The Contractor delivered work products or provided support in accordance with the SOW 90-94% on time with 6-10% of the product or support requiring rework.

The Contractor exceeded the delivered work product requirements or provided support in accordance with the SOW.

Product was delivered at least 95% on time with no more than 5% of the product or support requiring rework.

Customer and Government subject matter experts, Monthly feedback

Per each event The results of this evaluation will be reflected in CPARs/PPIRS.

Favorable consideration of One Year Option Year if score is greater than 90 with no score less than satisfactory.

3.11 Initial

Fielding Phase

The Contractor failed to deliver work products or provide support in accordance with the SOW below 90% on time with more than 10% of the product or support requiring rework.

The Contractor delivered work products or provided support in accordance with the SOW 90-94% on time with 6-10% of the product or support requiring rework.

The Contractor exceeded the delivered work product requirements or provided support in accordance with the SOW.

Product was delivered at least 95% on time with no more than 5% of the product or support requiring rework.

Customer and Government subject matter experts, Monthly feedback

Monthly The results of this evaluation will be reflected in CPARs/PPIRS.

One Year Option Year if score is greater than 90 with no score less than satisfactory.

UNCLASSIFIED // FOR OFFICIAL USE ONLY

3.11 DEVOPS The Contractor

failed to deliver work products or provide support in accordance with the SOW below 90% on time with more than 10% of the product or support requiring rework.

The Contractor delivered work products or provided support in accordance with the SOW 90-94% on time with 6-10% of the product or support requiring rework.

The Contractor exceeded the delivered work product requirements or provided support in accordance with the SOW.

Product was delivered at least 95% on time with no more than 5% of the product or support requiring rework.

Customer and Government subject matter experts, Monthly feedback

Monthly The results of this evaluation will be reflected in CPARs/PPIRS.

One Year Option Year if score is greater than 90 with no score less than satisfactory.

3.12 Annual

Sustainment Phase

The Contractor failed to deliver work products or provide support in accordance with the SOW below 90% on time with more than 10% of the product or support requiring rework.

The Contractor delivered work products or provided support in accordance with the SOW 90-94% on time with 6-10% of the product or support requiring rework.

The Contractor exceeded the delivered work product requirements or provided support in accordance with the SOW.

Product was delivered at least 95% on time with no more than 5% of the product or support requiring rework.

Customer and Government subject matter experts, Monthly feedback

Monthly The results of this evaluation will be reflected in CPARs/PPIRS.

Favorable consideration of One Year Option Year if score is greater than 90 with no score less than satisfactory.

3.20 Licensing The Contractor

failed to deliver work products or provide support in accordance with the SOW below 90% on time with more than 10% of the product or support requiring rework.

The Contractor delivered work products or provided support in accordance with the SOW 90-94% on time with 6-10% of the product or support requiring rework.

The Contractor exceeded the delivered work product requirements or provided support in accordance with the SOW.

Product was delivered at least 95% on time with no more than 5% of the product or support requiring rework.

Customer and Government subject matter experts, Monthly feedback

Monthly The results of this evaluation will be reflected in CPARs/PPIRS.

Favorable consideration of One Year Option Year if score is greater than 90 with no score less than satisfactory.

3.21 Configuration Management

The Contractor failed to deliver work products or provide support in accordance with the SOW below 90% on time with more than 10% of the product or support requiring rework.

The Contractor delivered work products or provided support in accordance with the SOW 90-94% on time with 6-10% of the product or support requiring rework.

The Contractor exceeded the delivered work product requirements or provided support in accordance with the SOW.

Product was delivered at least 95% on time with no more than 5% of the product or support requiring rework.

Customer and Government subject matter experts, quarterly feedback

Monthly The results of this evaluation will be reflected in CPARs/PPIRS.

One Year Option Year if score is greater than 90 with no score less than satisfactory.

All deliverables submitted to the Government within the timeframes stated in the DD Form 1423s or in

PWS

Less than 85% of all deliverables submitted to the Government within the timeframes stated in the DD Form 1423s.

Late deliverables submitted more than one month after the timeframes stated in the DD Form 1423s.

86-95% of all deliverables submitted to the Government within the timeframes stated in the DD Form 1423s. The remaining (i.e.

late deliverables) submitted no later than two weeks after the timeframes

96-100% of all deliverables submitted to the Government within the timeframes stated in the DD Form 1423s. The remaining (i.e.

late deliverables) submitted no later than one week after the timeframes stated in the DD Form 1423s.

100% of

COR and Government Team will assess all deliverables for timeliness, per the timeframes stated in the DD Form 1423s. If the Government grants an extension, the COR and Government team will assess the timeliness based upon the

As required. The results of this evaluation will be reflected in CPARs/PPIRS.

One Year Option Year if score is greater than 90 with no score less than satisfactory.

UNCLASSIFIED // FOR OFFICIAL USE ONLY

stated in the DD Form 1423s. 95% of any deliverables that are granted an extension by the Government are submitted by the extension date.

any deliverables that are granted an extension by the Government are submitted by the extension date.

extended submission date.

7. Performance Reporting

7.1. Performance Assessment Report

The COR will perform quarterly assessments and will file an annual assessment via the Contractor Performance Assessment Reporting System (CPARS). The Performance Assessment Report (PAR) provides the Government and the Contractor the ability to track the progress of Contractor performance in a formalized way to assist in the annual assessments submitted via CPARS. The PAR will be used to report all minor discrepancies and will be generated by the COR and provided to the PCO to be sent to the Contractor for corrective action. Contractors will be given 10 days to correct the items stated in the PARs, unless they have requested an extension and the extension has been granted. If three (3) or more minor discrepancies are found, the COR will combine them and process a Corrective Action Report (CAR).

Although the COR prepares the PAR quarterly, the Contractor’s performance will be monitored continuously and submitted to the PCO via monthly reports. The COR will review all deliverables required and submitted by the Contractor during various periods of contract performance to ensure compliance with the terms and conditions of the contract.

The COR will assist the PCO in monitoring contract performance per the COR appointment letter provided to the COR by the PCO.

The following is the PAR Form that will be used quarterly for evaluation of activities during the period of performance.

PERFORMANCE ASSESSMENT REPORT (PAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACT/TASK ORDER NUMBER 2. CONTRACTOR 3. TYPE OF SERVICES

4. QUALITY ASSURANCE PERSONNEL (COR) SIGNATURE AND DATE 5. COR PHONE 6. SUSPENSE DATE

I. PERFORMANCE

UNCLASSIFIED // FOR OFFICIAL USE ONLY

7. DEFICIENCY (CHECK ALL BOXES THAT APPLY)

NEW

REPEAT

NO DEFICIENCY NOTED

8. SERVICES SUMMARY or SOW PARAGRAPH ITEM REVIEWED

9. BRIEF DESCRIPTION OF DEFICIENCY (IF DEFICIENCY BOX WAS

CHECKED)

10. DETAILED PERFORMANCE ASSESSMENT

II. CONTRACTOR VALIDATION

11. CONTRACTOR REPRESENTATIVE CONCUR NON-CONCUR 12. CORRECTIVE ACTION ESTIMATED COMPLETION DATE

13. CONTRACTOR REPRESENTATIVE CORRECTIVE ACTION AND PREVENTION OF RECURRENCE OR REASON FOR NON-

CONCURRENCE OF COR CITED DEFICIENCY

III. ACTION CORRECTED

14. CONCUR NON-CONCUR COR SIGNATURE AND DATE

15. COR REMARKS (REQUIRED)

6. CONTRACTOR REPRESENTATIVE REMARKS

Quarterly assessment will be used for Annual assessment with the following evaluation factors:

a. Quality of Product or Service (Technical Performance) – The COR will assess the Contractor’s effort to transform operational needs and requirements into an integrated solution. Areas of focus may include the planning and management of program tasks, the quality of support provided throughout all phases of contract execution, and the management of a totally integrated effort of all program management concerns to meet cost, performance, and schedule objectives.

The Contractor will be evaluated quarterly with respect to the quality of the required support services and system enhancements using the standards, AQLs, and monitoring methods as outlined in this document. The COR will document and archive the evaluations in the surveillance folder. The Contractor’s contribution in meetings and reviews, quality of technical reports, productivity level, and the overall quality of technical support will be assessed within the evaluations.

b. Schedule and Timeliness of Deliveries – The COR will assess the Contractor’s adherence to the required delivery schedule by assessing the Contractor’s efforts during the assessment period that contribute to or affect the schedule variance. This will take into consideration the significance of scheduled events and assess the effectiveness of corrective actions taken by the Contractor.

The Contractor will be rated on the responsiveness to the contract tasks and their timely submissions of required support services and system enhancements, as required by the contract. In addition, the Contractor will be rated on their conduct of scheduled reviews and their timely submission of review materials, per the agreed

UNCLASSIFIED // FOR OFFICIAL USE ONLY

upon process and associated timelines as required by the contract.

c. Business Relations or Management – The COR will assess the turnaround, completeness, and quality of problem identification, corrective action plans, the cooperativeness of the Contractor, and customer satisfaction. The contractor’s success with management of subcontracts will be assessed by the COR. The Contractor’s ability to identify and apply resources required to meet schedule requirements, assign responsibility for tasks/actions required by the SOW, and communicate appropriate information to affected program elements in a timely manner will also be assessed. The COR will assess the Contractor’s program management and identified technical risks and the associated risk mitigation plans to determine if there are any impacts to PM DCGS-A’s project objectives. Contractor management should put adequate systems in place to most efficiently perform the contract and protect Government interests. For example, the Contractor should have approved Government property management, quality control, subcontracting, and accounting systems in place.

7.2. Customer Complaint Form

The COR will furnish instructions for this form to the Customer if the Customer pursues on filing a complaint of a service or product not meeting the Customer’s expectations. In addition, the Customer will receive the contract requirements that are associated with the service or product. The Customer must provide supporting evidence of complaint or feedback. After the COR validates feedback or complaint demonstrating Contractor noncompliance, the COR will process a PAR or CAR to the PCO. All Customer Complaint forms will be validated by the COR prior to notifying the Contractor.

CUSTOMER COMPLAINT RECORD

DATE/TIME OF COMPLAINT

SOURCE OF COMPLAINT

ORGANIZATION BUILDING NUMBER INDIVIDUAL PHONE NUMBER

NATURE OF COMPLAINT

CONTRACT REFERENCE

VALIDATION

UNCLASSIFIED // FOR OFFICIAL USE ONLY

DATE/TIME CONTRACTOR INFORMED OF COMPLAINT

ACTION TAKEN BY CONTRACTOR

RECEIVED/VALIDATED BY

7.3. Corrective Action Report (CAR)

The CAR will be used to report a situation when there are three or more minor discrepancies, or any major discrepancies documented. The CAR will be generated by the COR and forwarded to the PCO for action. A copy will be provided to the Contractor representative by the PCO. CARs are required to be corrected within 30 calendar days.

Depending on the complexity of the finding, the Contractor may be allowed up to six (6) months to correct a CAR. The COR will request that the Contractor provide the status of CARs monthly. The COR will keep an electronic log of all CARs and will keep a hard copy of CARs in the contract surveillance file.

CORRECTIVE ACTION REPORT (CAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACTOR 2. CONTRACT NUMBER 3. TYPE OF SERVICES

4. FUNCTIONAL AREA 5. SUSPENSE

DATE

6. CONTROL NUMBER

7. DEFICIENCY MAJOR MINOR

FINDING IMPACT:

FINDING:

Please respond with a written corrective action plan that details the corrective action of the cited deficiency, the cause of the deficiency, and actions taken to prevent recurrence by Suspense Date in Block 5. If date was not entered in Block 5, the contractor is not required to provide a response.

8. QUALITY ASSURANCE PERSONNEL (COR)

TYPED NAME AND GRADE SIGNATURE AND DATE

9. ISSUING AUTHORITY

UNCLASSIFIED // FOR OFFICIAL USE ONLY

TYPED NAME AND GRADE SIGNATURE AND DATE

10. COR RESPONSE TO CONTRACTOR CORRECTIVE ACTION AND ACTION TAKEN TO PREVENT RECURRENCE

11. COR DETERMINATION

ACCEPTED REJECTED

12. CLOSE DATE