Atch 3 2024.01.17 AMP TO 01 QASP.pdf

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

For

Analysis of Mobility Platform (AMP)

17 January 2024

APPROVED:

MAJ ALICIA M CARTER AMY M. MILLER

Program Manager Contracting Officer

USTRANSCOM/TCAQ-M USTRANSCOM/TCAQ-D

UNCLASSIFIED

TABLE OF CONTENTS:

1. INTRODUCTION

2. PURPOSE

3. ROLES AND RESPONSIBILITIES

3.1. Contracting Officer (KO)

3.2. Contract Specialist (CS)

3.3. Contracting Officer’s Representative (COR)

4. METHODS OF SURVEILLANCE

4.1. Sampling

4.2. Third Party Audits

4.3. Inspection/Review

4.4. Customer Inputs

4.5. Management Review

4.6. Non-SDS Items

5. EVALUATION METHODOLOGY

5.1. Established Procedures

5.2. Conversation Record

6. GOVERNMENT FURNISHED PROPERTY/EQUIPMENT

7. DOCUMENTATION

8. COMBATING TRAFFICKING IN PERSONS

9. LIST OF ACRONYMS

ATTACHMENT 1 – SERVICE DELIVERY SUMMARY (SDS)

ATTACHMENT 2 – PERFORMANCE ANALYSIS DISCREPANCY STATUS REPORT

1. INTRODUCTION

This Quality Assurance Surveillance Plan (QASP) has been developed to set forth the procedures and guidelines to be used by the Government to ensure that the contractor achieves the required performance standards as specified in the Performance Work Statement (PWS). The PWS contains a Service Delivery Summary (SDS) that summarizes performance objectives (service required) and performance thresholds (specific standard) contained within the body of the PWS. The SDS does not identify every service required, but only those services considered most important for mission accomplishment. This QASP is based on the premise that the contractor, and not the Government, is responsible for management and quality control actions to meet the terms of the contract. The SDS recognizes that the contractor is not a perfect manager and that unforeseen and uncontrollable problems do occur. Good management and use of an adequate quality control plan will allow the contractor to meet or exceed the performance standards specified in the

PWS.

2. PURPOSE

This QASP is designed to accomplish the following:

• Ensures the Government pays only for the level of services received

• Provides a systematic method to survey and evaluate contractor services to determine conformity with the technical requirements of the contract

• Defines the roles and responsibilities of participating Government officials

• Describes the evaluation methods that will be employed by the Government in assessing the contractor’s performance

• Describes the process of performance documentation

• Describes how the COR will monitor the contractor’s performance regarding trafficking in persons, IAW DFARS 222.1703 and PGI 222.1703(4)

• Describes the Government’s furnished property inventory process, if applicable, IAW

DFARS PGI 245.103-70 and FAR 52.245-1(f)(1)(iv)

• Provides copies of the Quality Assurance (QA) monitoring forms that will be used by the Government in documenting and evaluating contractor’s performance

3. ROLES AND RESPONSIBILITIES

The Government monitors contractor performance on a continuing basis through the services of a surveillance team. The surveillance team consists of the following members:

Contracting Officer (KO) and Contracting Officer’s Representative (COR).

3.1. Contracting Officer (KO)

The KO has overall responsibility for contract administration. The KO is responsible for monitoring contract compliance and resolving any and all disagreements regarding interpretation of contract terms and conditions. The two primary functions of the KO with regard to the surveillance activity are to approve payments and authorize changes to the contract. Payment is authorized based on documented objective evidence provided by the

COR that contractor performance meets the performance standards specified. The KO is the only Government office authorized to revise the contract. Additionally, the KO is responsible for approving this QASP.

3.2. Contract Specialist (CS)

The CS acts as an acquisition consultant and serves as liaison between the KO, the COR, and the Contractor.

3.3. Contracting Officer’s Representative (COR)

• The COR serves as a functional expert and is responsible for monitoring, assessing, recording, and reporting the technical performance of the contractor on a continuous basis.

• The COR is designated in writing by the KO to act as his or her authorized representative to assist in administering a contract. The COR’s limitations are contained in the written appointment letter.

• CORs are sufficiently trained to perform the required duties and to ensure their knowledge of the terms and conditions of the contract.

• The COR schedules surveillance activities, evaluates and documents performance by the contractor, initiates requests for and evaluates adequacy of the corrective action, and reports contractor performance of contractor requirements.

• The COR is to be objective, fair, and consistent in evaluating contractor performance against standards.

• The COR will notify the KO immediately when an evaluation shows that performance does not meet the standards identified in this QASP.

• The COR is required to ensure changes in work are not initiated before a written authorization or modification is issued by the KO.

• The COR is the certifying official, authorized to accept delivery of services.

• The COR must ensure that all services on all CLIN’s have been performed. Defense

Contract Audit Agency (DCAA) will enter the quantity received and digitally sign the receiving report/invoice in Wide Area Work Flow (WAWF). The COR/DCAA has a maximum of seven (7) days after the contractor’s submission of a properly documented receiving report/invoice to accept the quantity and digitally sign the document in WAWF. Prompt processing of receiving reports/invoices in WAWF increases the Government’s ability to take discounts offered and decreases the likelihood of the Government incurring interest expense for late payment. After digital signature, the signed receiving report/invoice will be routed to Defense Finance and Accounting Service (DFAS) for scheduling of payment. For Labor Hour Contract Line Item Numbers (CLINs), the COR must review on a monthly basis, the cost vouchers submitted and ensure the dollars expended are commensurate with the progress being made on the contract.

• The COR is required to provide an assessment of contractor performance to the KO for input into Contractor Performance Assessment Reporting System (CPARS).

4. METHODS OF SURVEILLANCE

Service Delivery Summary (SDS): The SDS summarizes the expected service objectives (outcomes) and identifies the metrics that will be tracked to determine whether the outcomes are being achieved at the appropriate levels of performance. The Government, through the COR, will monitor contractor performance using the surveillance method(s) described below. In determining the evaluation criteria, the Government has considered what the contract specifically calls for, how performance can be surveyed, and if the proposed method of surveillance is adequate to assure the required level of performance has been achieved.

Additionally, contractor performance will be a factor affecting application of the plan, i.e., surveillance frequency may be increased or decreased based on contractor demonstrated and documented performance. One or a combination of the following surveillance methods will be used to produces a well-rounded indication of contractor conformance: Sampling (spot, periodic, random), third party audit, inspection/review (verification of specific tasks, weekly, monthly, quarterly, biannual, and annual), and customer/Government input. Contractor will have no more than one late cyber-incident report or unreported cyber incident in a twelve (12)-month period.

4.1. Sampling

This is the most appropriate method for frequently recurring tasks. Random sampling is done to determine whether to accept or reject the contractor’s performance of the total lot of a particular task for a given period of time, using the premise that the statistically selected sample is representative of the entire lot. Sampling may be spot, periodic, or random.

4.2. Third Party Audits

Conducted by an authorized agent of the Government, federal, state, or local agencies (i.e., Occupational Safety and Health Administration (OSHA), Department of Motor Vehicles (DMV), Environmental Protection Agency (EPA), Environment Management, etc.).

4.3. Inspection/Review

This surveillance type is preferred for those tasks that occur infrequently. It is also used frequently for those tasks having very stringent performance requirements. When this type of surveillance is used, the COR must inspect and evaluate the contractor’s performance each time it is performed to determine acceptability. This type of surveillance consists of the evaluation of samples selected on other than a 100 percent or statistically random basis. The results of periodic surveillance inspections may be used as the basis for actions against the contractor. In such cases, the Inspection of Services clause becomes the basis for the KO’s actions. This will be done more frequently at the beginning of the contract and is expected to decrease as standards are consistently met, but may increase if performance falls below standard. The method for cyber security will be 100% inspection of all cyber incidents and reports.

4.4. Customer Inputs

The COR may use customer input to document discrepancies in contractor performance.

The KO may use validated customer complaints as the basis for actions against the contractor. In such cases, the appropriate contract clause addressing inspection of contractor performance becomes the basis for the KO’s actions. Under the philosophy of performance-based service contracting, we expect the customer to file complaints directly with the contractor with an automatic copy going to the COR. This allows COR oversight of the contractor’s progress in answering complaints, resolving problems, and updating the Quality Control (QC) program, while the contractor is responsible to the customer. When used, customer complaints must follow a formalized procedure:

4.4.1. The COR is the primary point of contact for and must receive copies of all customer complaints. Air Force (AF) Form 714, Customer Complaint Record, or a locally devised form may be used, but all complaints and resulting resolution must be documented with the information required on AF Form 714. Customer complaint forms become a permanent part of the COR surveillance records.

4.4.2. The COR will check customer complaints to ensure resolution of the deficiency and revision of the QC program to prevent recurrence.

4.5. Management Review

Methods of surveillance can change after contract award based on acceptance of a contractor’s proposed QC program or agreement that establishes the metrics to be used. When metrics are used as a method of surveillance through the partnering process, they may be developed after contract award, but prior to the performance start date.

4.6. Non-SDS Items

For required tasks not shown on the SDS, including all other tasks in the PWS and any referenced documents, the Government still retains the right to inspect any item included in the contract in accordance with Federal Acquisition Regulation (FAR) Part 46 Inspection of Services clause. Inspection of these services will be performed in the same general manner a periodic surveillance items mentioned above. The results of these inspections are documented, and if necessary, are provided to the KO for action. Should a discrepancy be observed, the KO will handle each documented discrepancy on a case-by-case basis.

5. EVALUATION METHODOLOGY

This QASP comprehensively guides the surveillance team’s activities and has been developed in a format to ensure ease of understanding and implementation. For each performance objective in the SDS, the specific method(s) of surveillance, performance thresholds (standards), sampling procedures, inspection procedures, and detailed objective task descriptors are shown in the SDS in Attachment 1. The methodology described in the chart shall be used as the basis for performing surveillance of the respective performance objectives.

5.1. Established Procedures

Each contractor assessment shall follow an established procedure for surveillance, recording, reporting, and follow-up, outlined as follows:

• The frequency of surveillance will be in accordance with the COR schedule

• Surveillance will be performed in accordance with Attachment 1 and results/comments recorded

• Performance/non-performance for a particular task will be entered chronologically by the COR on a historical log of surveyed performance. If no deficiency exists in contractor performance for this element, no further action is required.

• If contractor performance is deficient, the course of action is dependent on the severity/impact/frequency of the non-performance. Re-performance is the first action to resolve deficiencies. When the COR determines a deficiency is not Government caused, an AF Form 802, Contract Discrepancy Report, or a locally devised form, is initiated. The COR completes blocks 1 through 6 of the form and sends it to the KO. The KO must evaluate the Contract Discrepancy Report and, if appropriate, sign and send it to the contractor. The contractor must complete blocks 9 and 10 according to the requirements of the contract and return it to the KO within five (5) calendar days of receipt. Upon receipt of the contractor’s response, the KO, in consultation with the COR, must evaluate the contractor’s response and take the appropriate action. The KO must document the evaluation in block 11 and action taken in block 12 on the Contract Discrepancy Report. The COR shall submit a Discrepancy Status Report (see Attachment 2) and attach a copy of all Contract Discrepancy Reports for the reporting period.

• When corrective action is reported by the contractor, the COR will follow up with additional surveillance to verify implementation.

5.2. Conversation Record

Conversation Record, DOD Optional Form 271 may be used throughout this process to effectively document program issues and concerns addressed with the contractor and Government points of contact. This provides a method to keep QA personnel informed of the status of issues and concerns.

6. GOVERNMENT FURNISHED PROPERTY/EQUIPMENT

The contractor shall perform an annual inventory of Government Furnished Property (GFP)/Government Furnished Equipment (GFE) in August of each year. The inventory shall be accomplished, recorded, and records provided to the COR within 10 days of completion.

The report should include a report of discrepancies, a report on the loss of GFP/GFE, and a report of the GFP/GFE physical inventory results.

7. DOCUMENTATION

All surveillance activities must be documented to provide the required audit trail to justify Government acceptance and payment. The Documented Audit trail of the Surveillance

(DAS) activities is required by FAR 46.104(c). Contract Administration Office Responsibilities, which states, “Maintain, as part of the performance records of the contract, suitable records reflecting, (1) the nature of Government contract quality assurance actions including, when appropriate, the number of observations made and the number and type of defects; and (2) decisions regarding the acceptability of the products, the processes, and the requirements, as well as action to correct defects.”

All documentation resulting from surveillance is made a permanent part of the contract file.

The COR must keep the documentation files during the term of the contract and either monthly or at the conclusion of the contract (as directed by the KO) transfer the files to the KO for inclusion in the official contract file. It is the responsibility of the COR to establish and maintain this information in a DAS folder.

The surveillance folder should include as a minimum:

• Contract, including modifications

• Task Orders (TO), including modifications

• QASP

• COR Letter of Designation

• Invoices

• Surveillance Records o Written report of all inspections and timelines of deliverables o Written report of any deficiency o Any other written documentation relating to contract performance

• COR validation contractor has notified its employees of the United States Government policy of Trafficking in Persons, in accordance with FAR 52.222-50

• Inventory records of GFP/GFE

• General Correspondence

8. COMBATING TRAFFICKING IN PERSONS

• IAW DFARS PGI 222.17, the COR shall pursue, as appropriate, the following methods of monitoring the contractor’s performance regarding trafficking in persons such that non-compliance with FAR clause 52.222-50 are brought to the immediate attention of the KO.

• Keep the lines of communication open with the contractor. Remind the contractor of the contractual responsibilities to notify the Government if the contractor receives notifications of any alleged violations to this policy or if actions have been taken against the contractor employees, subcontractors, or subcontractor employees pursuant to the clause

• When appropriate, encourage contractor to complete Human Trafficking Awareness Training

• Encourage the contractor to take steps to investigate and eliminate slavery and human trafficking in their supply chains and to publish information for consumer awareness.

• Periodically access the Department of State’s Trafficking in Person (TIP) website at http://wwww.state.gov/g/tip for updates and to view the latest reports

• The COR will monitor contractor compliance with Federal Acquisition Regulations (FAR) Clause 52.222-50, Combating Trafficking in Person as follows:

o The COR will reiterate the notification requirements in Paragraph (d) of the clause to the contractor within 10 days after contract award.

o The COR will validate the contractor has notified its employees of 1) The United States Government’s zero tolerance policy described in paragraph (b) of FAR Clause 52.222-50; and 2) the actions to be taken against employees for violation in writing and provide a copy to the contracting officer.

9. LIST OF ACRONYMS

AF Air Force AMP Air Mobility Platform CLIN Contract Line Item Number COR Contracting Officer’s Representative CPARS Contractor Performance Assessment Reporting System DAS Documented Audit trail of the Surveillance DCAA Defense Contract Audit Agency DFAS Defense Finance and Accounting Service DISA Defense Information Systems Agency DMV Department of Motor Vehicles DOD Department of Defense EPA Environmental Protection Agency FAR Federal Acquisition Regulations GFE Government Furnished Equipment GFP Government Furnished Property IMS Integrated Master Schedule KO Contracting Officer ORR Operational Readiness Review OSHA Occupational Safety and Health Administration POA&Ms Plan of Actions and Milestones PRR Production Readiness Review PWS Performance Work Statement QA Quality Assurance QASP Quality Assurance Surveillance Plan QC Quality Control RDD Requirements and Design Document RMP Risk Management Plan SDS Service Delivery Summary

SRS Software Requirement Specification STD Software Test Description STIG Security Technical Implementation Guides SVD Software Version Description TO Task Order UAT User Acceptance Testing WAWF Wide Area Work Flow

ATTACHMENT 1 – SERVICE DELIVERY SUMMARY (SDS)

The SDS represents the most critical contract performance objectives that, when met, will ensure contract performance is satisfactory. Although not all PWS requirements are listed in the SDS, the contractor shall comply with all requirements in the PWS.

Table 1 Service Delivery Summary (SDS)

Performance Objective

PWS

Paragraph

Performance Threshold

Contract Management documents, and the Requirements Management Plan (RMP)

1.4.1 1.4.3 1.4.9

No more than one (1) late planned version of any of these documents over the course of the Task Order. All late documents must be delivered within five (5) days beyond the dates specified in the deliverable table, according to contractor- and Government-agreed schedule.

Quality: Final documents shall not contain more than 1 critical or more than 4 substantial comments per document, IAW Table 1, Comment Type.

Revisions/Drafts: No more than 2 sets of revisions/corrections with all revisions/corrections accomplished or incorporated by due date per document.

Note: A second “set” may be necessary should Government comments be required on the second draft (or “final”) delivery in response to the first set of Government comments.

System Releases 1.4.4.2 100% of software shall pass TCJ6 and Government selected Cloud Service Provider security scans, prior to Government acceptance

Algorithm Document, Requirements Definition Document (RDD), Software Design Document, Database Schema, Software User Manuals, Training Materials, and Training Manuals

1.4.4 1.4.4.2 1.4.4.3 1.4.5 1.4.10 1.4.11

No more than one (1) planned version of any of these documents delivered more than five (5) days beyond the dates specified in the deliverable table over the course of the delivery cycle, according to contractor- and Government-agreed schedule.

Quality: Final documents shall not contain more than 1 critical or 4 substantial comments IAW the Table 1, Comment Type, 100% of the time.

Software Deliverables ( Software Releases, Source Code)

1.4.4.2 1.4.4.3 1.4.10

No more than one (1) planned version of any of the software deliverables more than five (5) days beyond Operational Readiness Review (ORR) per delivery cycle, according to contractor- and Government-agreed schedule.

Quality: No more than

- Two (2) Priority 1 problem reports

- Four (4 ) Priority 2 problem reports

- Six (6) Priority 3 or below problem reports

- All Priority 1 and 2 and 50% or more of the Priority 3 problem reports submitted during User Acceptance Testing (UAT) resolved and operational software delivered on time, according to contractor-and Government-agreed schedule.

PRR: No more than 1 planned version delivered more than 2 days beyond Production Readiness Review (PRR), according to contractor- and Government-agreed schedule.

Security Documentation (Defense Information Systems Agency (DISA) Security Technical Implementation Guides (STIG) Checklist, Application Code Scan files, Plan of Action and Milestones (POA&Ms), etc)

1.4.7.2.2 1.4.7.1.3

No more than one (1) planned version of any of these documents delivered more than five (5) days beyond the dates specified in the deliverable table over the course of a software delivery cycle, according to contractor- and Government-agreed schedule.

Quality: Final documents shall not contain more than 1 critical or 4 substantial comments IAW Table 1, Comment Type, 100% of the time.

Revisions/Drafts: No more than 2 sets of revisions/corrections with all revisions/corrections accomplished or incorporated by due date per document.

Note: A second “set” may be necessary should Government comments be required on the second draft (or “final”) delivery in response to the first set of Government comments.

For clarity, Critical, substantial, and administrative comments are defined via the table below.

Table 2 Comment Type

Comment Type Definition Critical Comments of a very serious nature that will preclude final approval and publication of the document.

Substantial Comments that have noticeable impact on the document’s quality or content. The document appears to be incorrect, misleading, confusing, or inconsistent with other sections.

Administrative Comments that address minor items such as typographical, format, and/or grammatical errors.

Priority 1, Priority 2, and Priority 3 problem reports are defined in Table 2:

Table 3 Priority Level Description

Priority Description 1 System or major component of the system is inoperable 2 System or major component of the system is inaccurate and there is no work around to the problem 3 System or major component of the system in inaccurate, but a work around exists; need to publish the work around 4 Inconvenience to the user 5 Minor inconvenience to the user or aesthetic

ATTACHMENT 2 – PERFORMANCE ANALYSIS DISCREPANCY STATUS

REPORT