Attachment 2 QASP.pdf

N00421-20-R-0002

Contract Surveillance Plan and Quality Assurance Surveillance Plan for Advanced

Software, Computer, and Network Development and Integration

1. Purpose

This requirement consists of both performance-based and non-performance-based Task Orders

(TOs); therefore, a Contract Surveillance Plan (CSP) and a Quality Assurance Surveillance Plan

(QASP) are both required. The CSP/QASP is used to ensure that the Contractor performs In

Accordance With (IAW) the performance standards and definitions/metrics and the Government receives the quality of services called for in the Contract.

This hybrid CSP/QASP is a Government developed and applied document used to ensure systematic Quality Assurance (QA) methods are used in the administration and surveillance of tasking under the Contract. The intent is to ensure the Contractor performs IAW the non-performance-based required Level Of Effort (LOE) and performance-based standards and definitions/metrics, and the Government receives the quality of services called for in each TO

Statement of Work (SOW)/Performance Work Statement (PWS).

Naval Air Warfare Center Aircraft Division (NAWCAD) Webster Outlying Field (WOLF)

Airborne Systems Integration (ASI) Division has a requirement to procure services in support of rapid development, design, prototyping, low-rate production, demonstration, test, integration, technical installation, transition through Initial Operational Capability (IOC), and maintenance in the domains of cyber; Maritime Domain Awareness (MDA); situational awareness; data management and dissemination; and other rapid capability software requirements. These services include, but are not limited to, project management; research; analysis; design; parts and material procurement; demonstration; test and evaluation; and user/maintainer training. The Contract will consist of one (1) five-year ordering period.

A properly executed CSP/QASP will assist the Government in achieving the objectives of this procurement.

2. Authority

Authority for issuance of this hybrid CSP/QASP is provided under Contract Section E -

Inspection and Acceptance that provides for inspection and acceptance of the services, products, and documentation called for in the Contract that will be executed by the Contracting Officer

(KO) or a duly authorized representative.

3. Scope

To fully understand the roles and responsibilities of the parties, it is important to first define the distinction in terminology between Quality Control Plan (QCP) and the CSP/QASP. The

Contractor, and not the Government, is responsible for the QCP, which defines the management and quality control actions necessary to meet the quality standards set forth in the Contract and resultant TOs.

The hybrid CSP/QASP provides Government surveillance oversight of the Contractor’s efforts to assure they are timely, effective, and delivering the results specified in the Contract and each TO.

4. Government Resources

The following definitions for Government resources are applicable to this plan:

Contracting Officer (KO): A person duly appointed with the authority to enter into a contract

(Procuring Contracting Officer (PCO)), or a person with the authority to administer contracts

(Administrative Contracting Officer (ACO)), and make related determinations and findings on behalf of the Government. The PCO for this Contract is Thomas E. Stann. The ACO will be designated in the resulting Contract. KOs are designated via a written warrant that sets forth limitations of authority.

Contracting Officer’s Representative (COR): An individual appointed in writing by the PCO to act as their authorized representative to assist in administering the Contract. The COR will be appointed in the resulting Contract. The limitations of authority are contained in the letter of appointment.

Alternate Contracting Officer’s Representative (ACOR): An individual appointed in writing by the PCO to act as their authorized representative, in the absence of the COR, to assist in administering the Contract. The ACOR will be appointed in the resulting Contract. The limitations of authority are contained in a written letter of appointment.

Government Project Lead (GPL): The COR designates an individual GPL to assist in administering of specific efforts under the Contract.

5. Responsibilities

The following Government resources will have responsibility for the implementation of this hybrid CSP/QASP:

KO: The KO (either the PCO or ACO) ensures performance of all necessary actions for effective

Contracting, ensures compliance with the terms of the Contract, and safeguards the interests of the U.S. in the contractual relationship. The KO assures the Contractor receives impartial, fair, and equitable treatment under the Contract. The KO is ultimately responsible for the final determination of the adequacy of the Contractor’s performance.

COR: The COR is responsible for technical administration of the Contract and assures proper

Government surveillance of the Contractor’s performance. The COR is not empowered to make any contractual commitments or to authorize any changes on the Government’s behalf. Any changes that the Contractor deems may affect Contract price, terms, or conditions shall be referred to the KO for action.

ACOR: The ACOR assists the COR for technical administration of the Contract and assures proper Government surveillance of the Contractor’s performance. The ACOR is not empowered to make any contractual commitments or to authorize any changes on the Government’s behalf.

Any changes that the Contractor deems may affect the Contract price, terms, or conditions shall be referred to the KO for action.

GPL: The GPL is responsible for assisting in administering specific efforts under the Contract.

A GPL cannot, in any manner, alter the scope of the Contract, make commitments, or authorize any changes on the Government’s behalf.

6. Methods of QA Surveillance

a. Contractor Performance Assessment Reporting System (CPARS): The marketplace for rapid development, design, prototyping, low-rate production, demonstration, test, integration, technical installation, transition through IOC and maintenance in the domains of cyber; MDA; situational awareness; data management and dissemination of software technologies services is very competitive. As such, the successful Offeror has a vested interest in the Government-generated CPARS under this Contract. Additionally, an evaluation using the CPARS format will be performed each year beginning one (1) year after award. This evaluation will be one (1) determinant in the exercising of future Contract Line

Item Number (CLIN) option periods and awarding future TOs. The Government will address the quality of product or service, schedule, cost control, business relations, management, utilization of Small Business (SB), and other important areas for this procurement. Some of the monitoring methods to be used will be Government observation, review of reports, customer feedback, and review of Contract Data Requirements Lists (CDRLs). As this information may affect future source selections throughout the Department of Defense

(DoD), as well as continuation of this tasking, the annual Government assessment will be used appropriately as an additional performance oversight and communication tool within the policies of this hybrid CSP/QASP.

b. CSP/QASP: This document will be used by the COR, ACOR, and GPLs in the technical and cost administration of this Contract. In addition to the below instructions, the form to be used for documentation of QA surveillance is the hybrid checklist provided herein.

7. Surveillance

Enclosures (1) through (3) set forth the performance standards, incentives, and surveillance methods for the Contractor, the COR/ACOR and GPLs while Enclosures (4) through (6) provides the checklist and means for the COR to document the results of the surveillance and will be provided to the KO.

8. Documentation

In addition to providing reports to the KO, the COR will maintain a complete QA file. The file will contain copies of all reports, evaluations, recommendations, and any actions related to the

Government’s performance of the QA function, including the originals of the QA Checklists. All such records will be maintained for the life of the Contract. The COR will forward these records to the KO at termination or completion of the Contract.

9. Enclosures

Enclosure (1): Quality Standards – by SOW (Non-Performance-Based)/PWS (Performance-Based)

Enclosure (2): CDRL Data Quality Standards (Non-Performance-Based and Performance-Based)

Enclosure (3): Incentives

Enclosure (4): CSP (Non-Performance-Based and Performance-Based)

Enclosure (5): Contract Surveillance by CDRL (Non-Performance-Based and Performance-Based)

Enclosure (6): Annual Surveillance Summary

Enclosure (1)

Source Selection Information 5

See FAR 2.101 and 3.104

ENCLOSURE (1): QUALITY STANDARDS - BY SOW (Non-Performance-Based)

Required Service Standard (Performance Standards)/Maximum Allowable

Degree of Deviation Requirement (AQL)

Method of Surveillance (Quality

Assurance)

Incentives

Development and

Integration Support IAW

SOW/PWS Para 3.3.1

Based upon feedback from the GPLs, 95% of the services and products provided must be deemed comprehensive, timely, and technically acceptable.

100% Inspection by COR/ACOR and/or GPL.

See Enclosure (3)

Enclosure (1)

Source Selection Information 6

ENCLOSURE (1): QUALITY STANDARDS - BY PWS (Performance-Based)

Required Service Standard (Performance Standards)/Maximum Allowable

Degree of Deviation Requirement (AQL)

Method of Surveillance (Quality

Assurance)

Incentives

Development and Integration

Support IAW SOW/PWS

Para 3.3.2

Based upon feedback from the GPLs, 95% of the services and products provided must be deemed comprehensive, timely, and technically acceptable.

100% Inspection by COR/ACOR and/or GPL.

See Enclosure (3)

Safety Support IAW

SOW/PWS Para 3.3.3

Based upon feedback from the GPLs, 95% of the services and products provided must be deemed comprehensive, timely, and technically acceptable.

100% Inspection by COR/ACOR and/or GPL.

See Enclosure (3)

Program and Project

Support IAW SOW/PWS

Para 3.3.4

Based upon feedback from the GPLs, 95% of the services and products provided must be deemed comprehensive, timely, and technically acceptable.

100% Inspection by COR/ACOR and/or GPL.

Enclosure (2)

Source Selection Information 7

ENCLOSURE (2): CDRL DATA QUALITY STANDARDS

Description Standard AQL Surveillance/Method/Measure Incentives

Presentation Material

A001

Submission is IAW the

Data Item Description

(DID) cited in the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Test Procedures A002 Submission is IAW the

DID cited in the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Test Plan A003 Submission is IAW the

DID cited in the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Software Development

Plan (SDP) A004

Submission is IAW the

DID cited in the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Management Plan A005 Submission is IAW the

DID cited in the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Phase-Out Transition

Plan A006

Submission is IAW the

DID cited in the CDRL.

A one-time deliverable at each

TO close. Must be on time, with no more than one (1) revision per submission.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Contract Summary

Report A007

Submission is IAW the format detailed in BLK 16 of the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

Source Selection Information 8

Trip/Travel Report A008 Submission is IAW the format detailed in BLK 16 of the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

DoD Information

Assurance Certification and Accreditation

Process (DIACAP) and

Risk Management

Framework (RMF)

Deliverable Data A009

Submission is IAW the

DID cited in the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Operations Security

(OPSEC) Plan A010

Submission is IAW the format detailed in BLK 16 of the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

NAVAIR Cybersecurity

Implementation Plan

A011

Submission is IAW the format detailed in BLK 16 of the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Scientific & Technical

Reports Summary, Monthly Technical

Report A012

Submission is IAW the

DID cited in the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Scientific & Technical

Reports Summary, Ad-

Hoc Technical

Reports/Memos A013

Submission is IAW the

DID cited in the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, Source Selection Information 9

Contracting Officer’s

Management Report

A014

Submission is IAW the format detailed in BLK 16 of the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Test/Inspection Report, Test Report A015

Submission is IAW the

DID cited in the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Accident/Incident Report

A016

Submission is IAW the format detailed in BLK 16 of the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Software Design

Description A017

Submission is IAW the

DID cited in the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Software Version

Description A018

Submission is IAW the

DID cited in the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Interface Requirements

Specification A019

Submission is IAW the

DID cited in the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, ACOR, and/or GPL

See Enclosure (3)

Interface Design

Description A020

Submission is IAW the

DID cited in the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, Source Selection Information 10

System/Subsystem

Design Description A021

Submission is IAW the

DID cited in the CDRL.

95% of required deliverable provided on time. No more than one (1) report per year may be inaccurate or require a rewrite.

100% Inspection by COR, Enclosure (3)

Source Selection Information 11

ENCLOSURE (3): INCENTIVES

The following incentives will apply to performance under this Contract.

Assessment Period Acceptable Performance Definition How Measured Incentives

Year 1: Not more than one (1) of the measurement areas rated below

“Satisfactory.” None of the measurement areas are rated below

“Marginal.”

End of the first assessment period using the CPARS format.

(+) Meet the acceptable performance definition as a condition for additional tasking. *

(-) Does not meet the acceptable performance definition as a condition for additional tasking.*

Year 2: Not more than one (1) of the measurement areas rated below

“Satisfactory.” None of the measurement areas are rated below

“Marginal.”

End of the second assessment period using the CPARS format covering the previous twelve (12) months.

(+) Meet the acceptable performance definition as a condition for additional tasking. *

(-) Does not meet the acceptable performance definition as a condition for additional tasking.*

Year 3: Two (2) or more of the measurement areas rated “Very

Good” or above. None rated below

“Satisfactory.”

End of the third assessment period using the CPARS format covering the previous twelve (12) months.

(+) Meet the acceptable performance definition as a condition for additional tasking. *

(-) Does not meet the acceptable performance definition as a condition for additional tasking.*

Year 4: Two (2) or more of the measurement areas rated “Very

Good” or above. None of the measurement areas are below

“Satisfactory.”

End of the fourth assessment period evaluation using the CPARS format covering the previous twelve (12) months.

(+) Meet the acceptable performance definition as a condition for additional tasking. *

(-) Does not meet the acceptable performance definition as a condition for additional tasking.*

Source Selection Information 12

Assessment Period Acceptable Performance Definition How Measured Incentives

Year 5: Two (2) or more of the measurement areas rated “Very

Good” or “Exceptional.” None of the measurement areas are rated

“Marginal” or below.

End of the fifth assessment period using the CPARS format covering the previous twelve (12) months.

(+) Meet the acceptable performance definition as a condition for satisfactory or better Final CPARS rating.

(-) Does not meet the acceptable performance definition which will result in a marginal or unsatisfactory final CPARS rating.

Final CPAR Review After completion of all TOs. (+) Meet the acceptable performance definition as a condition for satisfactory or better Final CPARS rating.

(-) Does not meet the acceptable performance definition` which will result in a marginal or unsatisfactory

Final CPARS rating.

* The Government reserves the right not to award Option CLINs if the Contractor does not meet the acceptable performance standards, and/or meet all regulatory requirements.

The SOW/PWS and corresponding tasks and subtasks (to include CDRL submissions) will be assessed focusing on the following:

Quality of Product or Service: Assess the Contractor’s effort to transform operational needs and requirements into an integrated solution. Areas of focus may include the planning and management of program tasks, the quality of support provided throughout all phases of contract execution, the integration of program management specialties, management of interfaces, and the management of a totally integrated effort of all program management concerns to meet cost, performance, and schedule objectives. Assess how successfully the Contractor meets program quality.

Source Selection Information 13

Schedule: Assess the Contractor’s adherence to the required delivery schedule by assessing the Contractor’s efforts during the assessment period that contribute to or affect the schedule variance. Also address significance of scheduled events (i.e., design reviews), discuss causes, and assess the effectiveness of Contractor corrective actions.

Cost Control: Assess the Contractor’s effectiveness in forecasting, managing, and controlling Contract cost. Is the Contractor experiencing cost growth or under-run? If so, discuss the causes and Contractor proposed solutions for the cost overruns. For

Contracts where task or Contract sizing is based upon Contractor provided person hour estimates, the relationship of these estimates to ultimate cost should be assessed. In addition, the extent to which the Contractor demonstrates a sense of cost responsibility, through the efficient use of resources in each work effort will be assessed.

Business Relations: Assess the timeliness, completeness, and quality of problem identification, corrective action plans, proposal submittals, the Contractor’s reasonable and cooperative behavior, effective business relations, and customer satisfaction.

Management: Assess the Contractor’s success with timely award and management of subcontracts. Discuss the extent to which the

Contractor discharges its responsibility for integration and coordination of all activity needed to execute the Contract; identifies and applies resources required to meet schedule requirements; assigns responsibility for tasks/actions required by the Contract; and communicates appropriate information to affected program elements in a timely manner. Assess the Contractor’s risk mitigation plans.

Utilization of SB: FAR Subpart 19.7 and 15 U.S.C. 637 contains statutory requirements for complying with the SB Subcontracting

Program. Assess whether the Contractor provided maximum practicable opportunity for SBs to participate in Contract performance consistent with efficient performance of the Contract, to include:

Alaska Native Corporations (ANCs)

Indian Tribes

Small Disadvantaged Businesses (which also includes ANCs and Indian Tribes)

Women Owned Small Businesses (WOSB)

HUBZone

Veteran Owned Small Business (VOSB)

Service Disabled Veteran Owned Small Business (SDVOSB)

Source Selection Information 14

Evaluation will focus on the metrics of actual small business utilization compared to the Contract required targets and the Offeror’s

SB Plan, included as part of their proposal response.

Other areas: Assess additional evaluation areas unique to the Contract or that cannot be captured elsewhere.

The evaluation ratings are as follows:

Exceptional Performance: Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

Very Good Performance: Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

Satisfactory Performance: Performance meets contractual requirements. The contractual performance of the element or sub-element contains some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

Marginal Performance: Performance does not meet some contractual requirements. The contractual performance of the element or sub-element being evaluated reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented.

Unsatisfactory Performance: Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub-element contains a serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

For non-performance-based TOs, hours delivered by each TO compared to the TO required hours will be evaluated and will influence the ratings awarded above. Cost and/or labor hour variances greater than 10% cannot earn an Exceptional or Very Good for Cost

Control evaluation criteria.

Enclosure (4)

Source Selection Information 15

ENCLOSURE (4): CSP

Quarterly Surveillance:

The COR will perform a quarterly assessment of quality, schedule, cost control, business relations, management, utilization of SB, and other areas as applicable utilizing the evaluation rating definitions listed in Enclosure (3).

QUARTERLY EVALUATION – BY SOW/PWS

Description Surveillance

Method/Measure Dates

Quality

Assessment

Schedule

Assessment

Cost Control

Assessment

Business Relations

Assessment

Management

Assessment

Utilization of

SB

Assessment

Other Areas

Assessment

Development and Integration

Support IAW

SOW/PWS

Para 3.3.1

100% inspection by COR, ACOR, and/or GPL

Development and Integration

Support IAW

SOW/PWS

Para 3.3.2

100% inspection

Safety Support

IAW

SOW/PWS

Para 3.3.3

100% inspection

Program and

Project Support

IAW

SOW/PWS

Para 3.3.4

100% inspection

Enclosure (5)

Source Selection Information 16

ENCLOSURE (5): CONTRACT SURVEILLANCE BY CDRL

Description Surveillance Method/Measure Dates

Due

Dates

Completed

No.

Submittals

No. of

Req’d

Rework

Percentage

On Time Comments

Presentation Material A001 100% inspection by COR, ACOR, Test Procedures A002 100% inspection by COR, ACOR, Test Plan A003 100% inspection by COR, ACOR, Software Development Plan (SDP)

A004

100% inspection by COR, ACOR, Management Plan A005 100% inspection by COR, ACOR, Phase-Out Transition Plan A006 100% inspection by COR, ACOR, Contract Summary Report A007 100% inspection by COR, ACOR, Trip/Travel Report A008 100% inspection by COR, ACOR, DoD Information Assurance

Certification and Accreditation

Process (DIACAP) and Risk

Management Framework (RMF)

Deliverable Data A009

Operations Security (OPSEC) Plan

A010

NAVAIR Cybersecurity

Implementation Plan A011

Scientific & Technical Reports

Summary, Monthly Technical

Report A012

Scientific & Technical Reports

Summary, Ad-Hoc Technical

Reports/Memos A013

Enclosure (5)

Source Selection Information 17

Description Surveillance Method/Measure Dates

Due

Dates

Completed

No.

Submittals

No. of

Req’d

Rework

Percentage

On Time Comments

Contracting Officer’s Management

Report A014

Test/Inspection Report, Test Report

A015

Accident/Incident Report A016 100% inspection by COR, ACOR, Software Design Description A017 100% inspection by COR, ACOR, Software Version Description A018 100% inspection by COR, ACOR, Interface Requirements Specification

A019

Interface Design Description A020 100% inspection by COR, ACOR, System/Subsystem Design

Description A021

Metric Summary: Current Period Cumulative

Total Number of Deliverables Submitted:

Total Number of Deliverables Submitted Late:

Total Number of Deliverables Requiring Revision:

Percentage of Deliverables Submitted Late:

Percentage of Deliverables Requiring Revision:

Enclosure (6)

Source Selection Information 18

ENCLOSURE (6): ANNUAL SURVEILLANCE SUMMARY

The COR will perform an assessment of overall Contract performance utilizing the CPARS evaluation rating definitions listed in

Enclosure (3). This assessment will be completed annually covering the previous twelve (12) months.

Description Surveillance Method/Measure Date Planned Date Completed Summary Rating/Results

Quality of Product or Service Review of Quarterly Ratings

Schedule Review of Quarterly Ratings

Cost Control Review of Quarterly Ratings

Business Relations Review of Quarterly Ratings

Management Review of Quarterly Ratings

Utilization of SB Review of Quarterly Ratings

Other Areas Review of Quarterly Ratings

Annual Surveillance Completed By: _______________________

Date Completed: ________________