TTS QASP updated 4-7-2021.docx

Enclosure (3)

Quality Assurance Surveillance Plan (QASP) Sustainment, Upgrade, and Development of Tactical Integrated Threat/Target Training Systems (TTS) Services (Service CLINs Only)

Dated: 9 March 2021

Prepared by Naval Air Warfare Center Weapons Division China Lake, CA

DISTRIBUTION STATEMENT D:

Distribution authorized to the Department of Defense and U.S. DoD contractors only (Export Controlled Data) (10 December 2016). Other requests shall be referred to NAWCWD DB46200.

DESTRUCTION NOTICE:

In accordance with the TACTICAL INTEGRATED THREAT/TARGET TRAINING SYSTEMS (TTS) the destruction notice should read as such, "For classified documents, DoD civilians or military personnel will follow procedures outlined in DoDM 5200.01, Volume 3, Enclosure 3; DoD Contractors will follow procedures outlined in DoDM 5220.22, Sections 11-19. For sensitive unclassified documents, destroy by any method that will prevent disclosure of contents or reconstruction of the document.

WARNING: This document contains technical data whose export is restricted by the Arms Export Control Act (Title 22, U.S.C., Sec 2751, et seq.) or the Export Administration Act of 1979 (Title 50, U.S.C., App. 2401 et seq.), as amended. Violations of these export laws are subject to severe criminal penalties. Disseminate in accordance with provisions of DoD Directive 5230.25.”

1. Purpose

This Quality Assurance Surveillance Plan (QASP) is a Government developed and applied document used to ensure systematic quality assurance methods are used in the administration of the performance based services under the planned contract. The intent is to ensure that the contractor performs in accordance with the Performance Work Statement (PWS) performance metrics and the Government receives the quality of services called for in the contract.

This requirement as defined in the contract PWS defines the performance work efforts under TTS to provide lethal capability and operational readiness primarily in support of the Naval Aviation Training Systems Program Management Activity (PMA-205) and additional Department of Defense (DoD) activities that support the training ranges. The contractor shall sustain, upgrade, and develop tactical integrated threat/target training systems in accordance with the PWS.

The contract PWS also includes the following services: studies, analyses, and site surveys; Tactical Combat Training Systems (TCTS) information systems software baseline sustainment; TCTS field-level maintenance of systems, components, and repair; systems installation, integration, testing, and interoperability; systems training; engineering, configuration management/data management, logistics, and info-share website; TARIF LAN operation and maintenance.

This requirement will be met by issuing task orders with PWSs. The contract will be for a 5-year period of performance. A properly executed QASP will assist the Government in achieving the objectives of this procurement.

2. Authority

Authority for issuance of this QASP is provided by Section E - Inspection and Acceptance, which provides for inspection and acceptance of the services and documentation called for in the contract, to be executed by the Contracting Officer or a duly authorized representative.

3. Scope

To fully understand the roles and responsibilities of the parties, it is important to first define the distinction in terminology between the Quality Control Plan and the Quality Assurance Surveillance Plan. In the Quality Control Plan, the contractor, and not the Government, is responsible for management and quality control actions necessary to meet the quality standards set forth by the contract.

The QASP on the other hand, is put in place to provide Government surveillance oversight of the contractor’s efforts to ensure that they are timely, effective, and are delivering the results specified in the contract and documented in the order.

4. Government Resources

The following definitions for Government resources are applicable to this plan:

Contracting Officer - A person duly appointed with the authority to enter into (Procuring Contracting Officer (PCO)) or administer (Administrative Contracting Officer (ACO)) contracts and makes related determination and findings on behalf of the Government. The PCO for this contract is Tara Brandt. The ACO will be designated upon award of the resulting contract. Contracting Officers are designated via a written warrant, which sets forth limitations of authority.

Contracting Officer’s Representative (COR) - An individual appointed in writing by the PCO to act as their authorized representative to assist in administering the contract. The COR will be appointed upon award of the resulting contract. The limitations of authority are contained in a written letter of appointment.

5. Responsibilities

The following Government resources shall have responsibility for the implementation of this QASP:

Contracting Officer – The Contracting Officer ensures performance of all necessary actions for effective contracting, ensures compliance with the terms of the contract and safeguards the interests of the United States in the contractual relationship. It is the Contracting Officer who ensures the contractor receives impartial, fair, and equitable treatment under the contract. The Contracting Officer is ultimately responsible for the final determination of the adequacy of the contractor’s performance.

Contracting Officer’s Representative – The COR is responsible for technical administration of the contract and ensures proper Government surveillance of the contractor’s performance. The COR is not empowered to make any contractual commitments or to authorize any changes on the Government’s behalf. Any changes that the contractor deems may affect contract price, terms, or conditions shall be referred to the Contracting Officer for action.

6. Methods of Quality Assurance Surveillance

a. Contractor Performance Assessment Reporting System (CPARS) – The Government will perform an evaluation using the CPARS format 1 month prior to the end of each order period of performance. The Government will address the quality of product or service, schedule, cost control, business relations, management, utilization of small business, and other important areas. This evaluation will be one determinant in performance of each sequential period. As this information may affect future source selections throughout DoD, and future orders, the Government assessment will be used appropriately as an additional performance oversight and communication tool.

b. QASP - The below listed methods of surveillance shall be used by the COR and the Government Project Leads in the technical administration of this QASP. In addition to the below instructions, the form to be used for documentation of quality assurance surveillance is the Surveillance Activity Checklist provided herein.

c. Observation, Feedback, and Review – Unscheduled observation of products and services provided by the contractor on site and recurring review of contractor submitted reports and documentation identified in the CDRLs will be performed by the COR. Feedback from TTS customers will be collected and used by the COR to document surveillance of technical performance.

7. Surveillance

Enclosures (1) and (2) set forth the performance standards, incentives, and surveillance methods for the contractor and COR while enclosure (3) provides the checklist and means for the COR to document the results of the surveillance and shall be provided to the Contracting Officer upon completion. The frequency will be as specified in the order.

8. Documentation

In addition to providing surveillance reports to the Contracting Officer, the COR will maintain a complete Quality Assurance file. The file will contain copies of all reports, evaluations, recommendations, and any actions related to the Government’s performance of the quality assurance function, including the originals of the Surveillance Activity Checklists. All such records will be maintained for the life of the contract. The COR shall forward these records to the Contracting Officer at termination or completion of the contract.

9. Enclosures

Enclosure (1) – Performance Quality Standards Enclosure (2) – Incentives Enclosure (3) – Surveillance Activity Checklist

Section J, Attachment 1 N68936-20-R-0106

PERFORMANCE QUALITY STANDARDS

Required Service (Performance Requirements)

(Quality Assurance) Incentives

Provide TCTS Information Systems Software Baseline Sustainment IAW PWS Para 3.1

The contractor shall provide TCTS Information Systems Software Baseline Sustainment IAW PWS Para 3.1. Based upon feedback from the COR and/or Government Project Leads, the services provided meet requirements and are delivered on time.

Initial testing validates the Software Problem Report (SPR) requirements are at least 85% accurate. Final Testing validates the SPRs requirements are 100% accurate.

100% Inspection by COR and/or Government Project Leads.

See Enclosure (2)

Provide Software Development, Modification, and Maintenance IAW PWS Para 3.2.

The contractor shall provide Software Development, Modification, and Maintenance IAW PWS Para 3.2. Based upon feedback from the COR and/or Government Project Leads, the services provided meet requirements and are delivered on time.

Initial testing validates the Software Problem Report (SPR) requirements are at least 85% accurate. Final Testing validates the SPRs requirements are 100% accurate or as required at the TO/DO level.

100% Inspection by COR and/or Government Project Leads.

See Enclosure (2)

Analyze and Validate Trusted Cross-Domain Solutions IAW PWS Para 3.3.

Configure, Install, integrate, and sustain cross-domain solutions IAW PWS Para 3.3.

The contractor shall analyze and recommend course of action to the acquisition of trusted Cross-Domain Solutions IAW PWS Para 3.3. Based upon feedback from the COR and/or Government Project Leads, the services provided meet requirements and are delivered on time.

Recommendations provided IAW CDRL delivery schedule at 95%.

The contractor shall program, install, integrate, and sustain cross-domain solutions IAW PWS Para 3.3.

Cross Domain Solution Office (CDSO) verifies package is complete (100%)

100% Inspection by COR and/or Government Project Leads.

See Enclosure (2)

Provide Test and Training Enabling Network Architecture and Threat Interfaces IAW PWS Para 3.4

The contractor shall provide Test and Training Enabling Network Architecture and Threat Interfaces IAW PWS Para 3.4. Based upon feedback from the COR and/or Government Project Leads, the services provided meet requirements and are delivered on time.

TENA and Threat Interfaces validated against interface documents (100% accurate)

100% Inspection by COR and/or Government Project Leads.

See Enclosure (2)

Feedback from TPOC indicates deliverables meet minimum requirements (100%) and schedule (95%).

100% Inspection by COR and/or Government Project Leads.

See Enclosure (2)

Provide Electronic Warfare Server and EW Multiplexer Integration IAW PWS Para 3.6.

The contractor shall provide Electronic Warfare Server and EW Multiplexer Integration IAW PWS Para 3.6.Based upon feedback from the COR and/or Government Project Leads, the services provided meet requirements and are delivered on time.

System testing verifies requirements are consistent with interface documentation (100% accuracy).

See Enclosure (2)

Provide Weapon and Threat Simulations IAW PWS Para 3.7

The contractor shall provide Weapon and Threat Simulations IAW PWS Para 3.8.. Based upon feedback from the COR and/or Government Project Leads, the services provided meet requirements and are delivered on time.

Testing validates specifications are met with (95% accuracy).

See Enclosure (2)

Provide Studies, Analyses and Site Surveys IAW PWS Para 3.8.

The contractor shall provide Studies, Analyses and Site Surveys IAW PWS Para 3.9. Based upon feedback from the COR and/or Government Project Leads, the services provided meet requirements and are delivered on time.

Delivered in accordance with order schedule. Feedback from TPOC indicate the documentation provided is relevant and complete.

100% Inspection by COR and/or Government Project Leads.

See Enclosure (2)

Provide Web-Accessible Database IAW PWS Para 3.9.

The contractor shall provide Web-Accessible Database IAW PWS Para 3.10. Based upon feedback from the COR and/or Government Project Leads, the services provided meet requirements and are delivered on time.

Database is maintained and available 24 hours per day, 7 days a week. The website is not down from more than 1 hour per week for maintenance. System down time or any other changes are communicated to the Government within a minimum of 24 hrs. (100% accuracy)

100% Inspection by COR and/or Government Project Leads.

See Enclosure (2)

Provide System Support for the Tactical Training Ranges IAW PWS Para 3.10.

Provide System Support for the Tactical Training Ranges IAW PWS Para 3.11. Based upon feedback from the COR and/or Government Project Leads, the services provided meet requirements and are delivered on time.

Engineering, configuration management/data management, and logistic support is provided IAW order requirements. Feedback from Ranges indicates requirements have been met (100%)

The contractor shall respond to trouble calls within 3 working days of received trouble calls. (95% accuracy)

Provide cybersecurity and System Administration IAW PWS Para 3.11.

The contractor shall provide Cybersecurity and System Administration IAW PWS para 3.12. Based upon feedback from the COR and/or Government Project Leads, the services provided meet requirements and are delivered on time.

Networks and servers are maintained and up to date. (100%) Documentation is accurate and complete and provided on time IAW with CDRL schedule (95%)

100% Inspection by COR and/or Government Project Leads.

See Enclosure (2)

Provide Training IAW PWS Para 3.12.

Provide training IAW PWS Para 3.13. Based upon feedback from the COR and/or Government Project Leads, the services provided meet requirements and are delivered on time.

Training provided IAW order requirements and deliverables. Feedback from 90% of the stakeholders indicates training is relevant, accurate, and timely.

100% Inspection by COR and/or Government Project Leads.

See Enclosure (2)

Initial testing validates the SPR requirements are at least 85% accurate. Final Testing validates the SPRs requirements are 100% accurate.

DATA PERFORMANCE STANDARDS

Description

Standard

AQL

Surveillance/ Method/Measure

Incentives

INCENTIVES

The following incentives shall apply to performance under this contract.

Incentives

(-) Does not meet the acceptable performance definition as a condition for additional task order awards.

All PWS/CDRL tasks, including PWS/CDRL sub-tasks, will be assessed focusing on the following:

Quality of Product or Service – Assess the contractor’s effort to transform operational needs and requirements into an integrated solution. Areas of focus may include the planning and management of program efforts, the quality of support provided throughout all phases of contract execution, the integration of program management specialties, management of interfaces, and the management of a totally integrated effort of all program management concerns to meet cost, performance, and schedule objectives. Assess how successfully the contractor meets program quality.

Schedule – Assess the contractor’s adherence to the required delivery schedule by assessing the contractor’s efforts during the assessment period that contribute to or effect the schedule variance. Also address significance of scheduled events (i.e., design reviews), discuss causes, and assess the effectiveness of contractor corrective actions.

Cost Control – Assess the contractor’s effectiveness in forecasting, managing, and controlling contract cost. Is the contractor experiencing cost growth or underrun? If so, discuss the causes and contractor-proposed solutions for the cost overruns. For contracts where task or contract sizing is based upon contractor-provided person-hour estimates, the relationship of these estimates to ultimate cost should be assessed. In addition, the extent to which the contractor demonstrates a sense of cost responsibility through the efficient use of resources in each work effort should be assessed.

Business Relations – Assess the timelines, completeness and quality of problem identification, corrective action plans, proposal submittals, the contractor’s reasonable and cooperative behavior, effective business relations, and customer satisfaction.

Management – Assess the contractor’s success with timely award and management of subcontracts, including whether the contractor met small/small disadvantage and women-owned business participation goals. Discuss the extent to which the contractor discharges its responsibility for integration and coordination of all activity needed to execute the contract; identifies and applies resources required to meet schedule requirements; assigns responsibility for tasks/actions required by contract; and communicates appropriate information to affected program elements in a timely manner. Assess the contractor’s risk mitigation plans. If applicable, identify any other management areas that are unique to the contract.

Utilization of Small Business – Assess the contractor’s success in utilizing and meeting small business goals/concerns.

Other areas – Assess additional evaluation areas unique to the contract, or that cannot be captured elsewhere.

The evaluation ratings are as follows:

Performance Based Evaluation: The Government will evaluate contractor performance based on their ability to complete work specified in the scope and PWS Paragraphs 3.1 through 3.14. The contractor shall be evaluated using the following ratings:

Exceptional Performance: Contractor’s performance of virtually all contract requirements consistently exceeds the acceptable standards and in addition provides numerous significant, tangible or intangible, benefits to the Government (e.g., improved quality, responsiveness, increased timeliness, or generally enhanced effectiveness of operations). The few areas for improvement are all minor. There are no recurring problems. Contractor’s management initiates effective corrective action whenever needed.

Very Good Performance: Contractor’s performance of most contract requirements is consistently above acceptable standards and provides significant tangible and intangible benefits to the Government (e.g., improved quality, responsiveness, increased timeliness, or generally enhanced effectiveness of operations). Although some areas require improvement, these areas are minor and are offset by better performance in other areas. Few, if any, recurring problems have been noted, and the contractor takes satisfactory corrective action.

Satisfactory Performance: Contractor’s performance of most contract requirements meets the acceptable standards with some tangible or intangible benefits to the Government (e.g., improved quality, responsiveness, increased timeliness, or generally enhanced effectiveness of operations). Although there are some areas of very good or excellent performance, these are more or less offset by unsatisfactory performance in other areas.

Marginal: Performance does not meet contractual requirements. The contractual performance of the task and sub-task being assessed reflect a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented.

Unsatisfactory Performance: Contractor’s performance of most contract requirements is inconsistent and frequently does not meet acceptable standards. Quality, responsiveness, and timeliness in many areas require attention and action. Corrective actions have not been taken or are ineffective.

Enclosure (1) Enclosure (2)

Enclosure (3)

SURVEILLANCE ACTIVITY CHECKLIST

Surveillance The COR will perform an assessment of Quality, Schedule, Cost Control, Business Relations, Management, Utilization of Small Business, and Other Areas as applicable utilizing the CPARS evaluation rating definitions listed in Enclosure 2.

Task Order Requirements in accordance with the Statement of Work

Measure

Summary of Ratings For:

Task Order requirements in accordance with the Performance Based Statement of Work.

Checklist Completed By: ____________________________

Date Completed: ____________________________

Task Order Requirements in accordance with Contract Data Requirements Lists (CDRLs) Form DD-1423

Measure

Summary of Ratings For:

Task Order Requirements in accordance with CDRLs Form DD-1423.

Checklist Completed By: ____________________________

Date Completed: ____________________________

Annual Surveillance:

The COR will perform an annual assessment (or another time period specified in the order) of overall task order performance utilizing the CPARS evaluation rating definitions listed in Enclosure 2.

Measure

Schedule Review of Ratings.

Cost Control Review of Ratings.

Business Relations Review of Ratings.

Management Review of Ratings.

Other Areas Review of Ratings.

Annual Surveillance Completed By: ____________________________

Date Completed: ____________________________