Section_J_Atch_16-QASP_for_APT_Program.pdf

QUALITY ASSURANCE SURVEILLANCE PLAN

for the

ADVANCED PILOT TRAINING PROGRAM

FA8617-17-R-6219

26 September 2016

Version 1.0

Prepared by

Air Force Life Cycle Management Center (AFLCMC) Training Aircraft Division (WLZ)

Advanced Pilot Training Branch (WLZG)

Solicitation FA8617-17-R-6219 Section J – Attachment 16

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- ii -

- iii -

PART I: PERFORMANCE MANAGEMENT

1 INTRODUCTION

2 OBJECTIVE

3 GOALS OF THE MULTI-FUNCTIONAL TEAM

4 ROLES AND RESPONSIBILITIES

4.1 Chief of the Contracting Office

4.2 Contracting Officer (CO)

4.3 Contracting Officer’s Representative Supervisor

4.4 Functional Commander / Functional Director (FC/FD)

4.5 Quality Assurance Program Coordinator (QAPC)

4.6 Contracting Officer’s Representatives

4.7 Subject Matter Experts (i.e., Legal, Finance, Small Business, and Engineering Points of Contact (POCs))

5 CONTRACT ASSESSMENT METHODS

5.1 Market Research

5.2 Multi-Functional Team Meetings

5.3 Data Review and Corrective Actions

6 PERFORMANCE ASSESSMENT

6.1 Contractor Responsibility

6.2 Approach

6.3 Services Summary (SS)

6.4 Quality Assurance Procedures

6.5 Contractor Performance Assessment Reporting System

6.6 Combating Trafficking in Persons

7 REVISIONS

8 SAFETY REQUIREMENTS

8.1 Mishap Notification

8.2 Mishap Investigations

8.3 Multi-Functional Team Surveillance

PART II: ASSESSMENT MANAGEMENT

1 INTRODUCTION

1.1 Inspection

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- iv -

1.2 Defects

1.3 Performance Responsibility

1.4 Approach

2 METHODS FOR SURVEILLANCE PERFORMANCE

2.1 Customer Complaint/Feedback

2.2 Periodic Surveillance

2.3 100 Percent Surveillance

3 INFORM CONTRACTOR’S CONTRACT MANAGER

4 EVALUATE CONTRACTOR CORRECTIVE ACTION

5 REMEDIES FOR UNACCEPTABLE PERFORMANCE

6 CONTRACTING OFFICER’S REPRESENTATIVE MONTHLY

INSPECTION REPORT

7 WIDE AREA WORKFLOW

7.1 Receipt Certification

7.2 Wide Area Workflow Training

8 REVISION TO THE QUALITY ASSURANCE SURVEILLANCE PLAN

9 PERFORMANCE OBJECTIVES, PERFORMANCE THRESHOLDS,

AND METHODS OF SURVEILLANCE

9.1 SS - 1 Satisfy Total Non-Mission Capable Supply Rate

9.2 SS - 2 Issue Effectiveness Rate

9.3 SS – 3 Aircrew Training Devices Certified for Training

9.4 SS – 4 Trainer Availability

9.5 SS – 5 Provide Accurate Resolutions to Customer Needs and Agency Requirements.. 21

9.6 SS – 6 Provide Organizational-Level Field Service Representative as part of Sustainment

MULTI-FUNCTIONAL TEAM MEMBERSHIP

CONTRACTING OFFICER REPRESENTATIVE SCHEDULE

COR SURVEILLANCE CHECKLIST

CUSTOMER FEEDBACK RECORD

CUSTOMER COMPLAINT RECORD

CORRECTIVE ACTION REPORT

NON-SERVICES SUMMARY (SS) INSPECTION FORM

ACRONYM LIST

- v -

QUALITY ASSURANCE

SURVEILLANCE PLAN

Part 1

PERFORMANCE

MANAGEMENT

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 1 -

1 INTRODUCTION

This Quality Assurance Surveillance Plan (QASP) has been developed to implement Air Force Instruction (AFI) 63-101/20-101, Chapter 8, Acquisition of Services of Performance-Based Service Acquisitions (PBSA), for the Advanced Pilot Training (APT) Program aircraft and Ground Based Training System (GBTS) procurement. The QASP is designed to assess contract success and to assess and document Contractor performance in accordance with (IAW) the requirements set forth in the APT contract for the production and sustainment phases. The specifics of this work performed on this contract are defined in the APT Statement of Work

(SOW).

The QASP also provides the Contracting Officer’s Representative (COR) an effective and systematic surveillance method for each listed service. In addition, this QASP implements the multi-functional team (MFT) concept. The goal of the MFT is to give all members/stakeholders the ability to maintain the highest quality-representative service to the user and propose/initiate improvements throughout the life of the contract. Under this concept, continual contract assessment will be focused on ensuring that the contract, as written, continues to meet the needs of the user. The signatures, on the attached MFT Membership List charter signature page, indicate membership in the MFT. Changes to team members will require an amendment to the charter signature page.

The QASP is published separately from and is not part of the contract. It is issued with the solicitation for informational purposes only. The QASP procedures can be changed unilaterally by the Government at any time.

In an effort to streamline the quality assurance (QA) provisions described in AFI 63-101, Integrated Life Cycle Management, this QASP has been developed to cover all applicable APT contracts with service-designated contract line item numbers.

2 OBJECTIVE

The APT Program will replace the T-38C used in the United States Air Force’s (USAF) Specialized Undergraduate Pilot Training advanced phase fighter/bomber track and in the Introduction to Fighter Fundamentals course. The APT Program will provide student pilots with the foundational flying skills and core competencies required to transition into current generation fighter/bomber aircraft.

The objective of this plan is to identify methods used for evaluation and surveillance of the quality of all applicable services provided by the Contractor throughout all production and sustainment activities, including Interim Contractor Support (ICS). Contractor logistics support for the APT GBTS, as identified in the APT SOW, is also included, but not limited to, Training System Support Center support (including configuration management, logistics support, Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 2 -integration, software modification, and validation of all GBTS Software) and the simulator maintenance activities for the operations, maintenance, and support for the APT GBTS.

The QASP details the USAF’s plan for monitoring the Contractor’s performance while providing oversight capability to make certain the Contractor is delivering products and services that conform to contract requirements. In addition, the QASP’s goal is to continually explore and implement accurate performance measurements reflecting the continual improvements needed to maximize customer service and result in lower costs, faster service, and higher quality. The COR will use a variety of surveillance and assessment methods to evaluate the Contractor’s performance. This QASP is not intended to duplicate the Contractor’s Quality Control Plan (QCP). This plan is a living document.

3 GOALS OF THE MULTI-FUNCTIONAL TEAM

The MFT should strive to achieve efficiencies in the contract performance and to achieve improved customer support. The team is composed of stakeholders in the acquisition process and is responsible for the acquisition throughout the life of the contract. The goal of the team is to define the requirement in performance-based terms, ensure Contractor compliance with contract requirements, and ensure that the contract, as written, continues to meet the needs of the functional organization. The team promotes good business decisions to meet customer requirements, fosters partnerships with industry to ensure exchanges of information with the Contractor and other business experts, develops a performance-based acquisition strategy to include performance metrics, develops PBSA and technical requirements, and develops the independent Government cost estimate. The team is also responsible for managing contract performance IAW the QASP, assessing and documenting Contractor performance IAW the QASP, identifying opportunities to improve performance throughout the life of the contract, and completing and reporting annual performance reviews in the Contractor Performance Assessment Reporting System (CPARS), as required. In addition, another goal is implementation and execution of the acquisition strategy to ensure support of the mission. The goal entails planning, programming, and budgeting for adequate funds, ensuring program execution within the approved funding, developing the milestones and managing their execution to ensure the contract supports mission requirements within the approved scheduled baseline, ensuring the program maintains accountability, and planning and management of contract services. After contract award, the APT Contractor may be invited to become a limited-participation member of the MFT (i.e., not involved in any inherently governmental responsibilities).

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 3 -

4 ROLES AND RESPONSIBILITIES

4.1 Chief of the Contracting Office

• Serves as Business Advisor to senior leadership.

• Updates the Program Manager, Program Executive Officer, AFLCMC Commander, and other headquarters directors annually on the health of the service acquisitions.

• Ensures personnel on the MFT receive the appropriate level of training to meet local needs.

• Ensures compliance with the Federal Acquisition Regulation (FAR) and supplements.

• Ensures functional area leadership receives training.

4.2 Contracting Officer (CO)

• Ensures acquisition processes (i.e., market research, past performance information gathering, source selection, solicitations issuance, proposal evaluation, and contract award) are aggressively performed.

• Participates in MFT meetings.

• Delegates authority for inspection and/or acceptance to the COR IAW the terms and conditions of the contract. Informs the Contractor of the names, duties, and limitations of authority for all CORs assigned to the contract.

• Advises the MFT on QASP development including incentives and remedies tied to performance objectives and performance thresholds IAW Air Force Federal Acquisition Regulation Supplement (AFFARS) MP5346.103-1.b.(1). Develops and awards a contract/order based on MFT requirements and acquisition strategy. Only individual authorized to enter into or change the contract for supplies and/or services by obligating the Government.

• Determines if a COR is required to conduct/document surveillance of Contractor performance IAW AFFARS MP5301.602-2(d)1.2, Designation, Assignment, and Responsibilities of a Contracting Officer’s Representative (COR), Revised 3 June 2016.

Appoints and approves COR nominations in the Contracting Officer Representative Tracking (CORT) Tool and submits the Designation of COR Memorandum to the COR and Contractor for signature, thereby ensuring the Contractor is notified of the COR assigned to the contract/order and his/her areas of responsibility and limitation of authority IAW Defense Acquisition Regulation Supplement (DFARS) PGI 201.602- 2(d)(v)(A) and AFFARS MP5301.602-2(d)1.5. Upon transfer/termination of the COR, issues a Revocation (Termination) of Designation of COR Memorandum IAW AFFARS MP5301.602-2(d)1.10-1.12. Uploads Designation of COR Memorandum and, if applicable, Termination/Revocation of Contracting Officer’s Representative Designation Memorandum in the CORT Tool IAW DFARS PGI 201.602-2(d)(v)(C), Responsibilities, and the Office of the Under Secretary of Defense (OUSD) Memorandum, dated 21 March 2011. Provides contract-specific COR training to the COR IAW AFFARS MP5301.602-2(d)1.4.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 4 -

• Reviews COR reports (e.g., Performance Assessment Reports (PAR), Corrective Action Reports (CAR)) on Contractor performance IAW AFFARS MP5301.602-2(d)1.6.

Inspects COR Online File and COR Profile in the CORT Tool, at a minimum, annually, IAW DFARS PGI 201.602-2(d)(vii). Reviews and signs the Annual COR Online File Inspection Checklist and uploads checklist in the CORT Tool IAW DFARS PGI 201.602-2(d)(vii).

• Responsible for administration of the contract/order except as delegated to the Administrative Contracting Officer. Takes action, if necessary, to enforce contractual requirements.

4.3 Contracting Officer’s Representative Supervisor

• Nominates a primary and alternate COR unless another QA organization or method is used to satisfy surveillance requirements (e.g., centralized performance management office).

• Ensures nomination package specifies nominee’s qualifications.

• Upon notification of the COR nomination in the CORT Tool (within Wide Area Workflow (WAWF) E-Business Suite), approves COR nomination in the CORT Tool IAW DFARS PGI 201.602-2(d)(iv) and the OUSD Memorandum, dated 21 March 2011. Inspects the COR Online File and the COR Profile in the CORT Tool, at a minimum, annually, IAW DFARS PGI 201.602-2(d)(vii). Reviews and signs the Annual COR Online File Inspection Checklist IAW DFARS PGI 201.602-2(d)(vii).

• Ensures the COR is afforded the necessary resources (i.e., time, supplies, equipment, opportunity) to perform the designated functions.

• Reviews the Contractor performance documentation prepared by the COR on a regular basis to ensure performance is compatible with contract and mission objectives.

• Evaluates the COR on performance of his/her COR duties – both pre- and post-award – as part of his/her performance assessment throughout the period of the contract.

• Solicits input on the COR’s performance of COR duties from the CO.

4.4 Functional Commander / Functional Director (FC/FD)

• Ensures requirements documents developed by the MFT satisfy mission requirements, are performance based, foster innovation, and define metrics. Metrics should provide meaningful measurements of the Contractor progress aimed at attaining desired outcomes.

• Identifies mission-essential services and develops the necessary documents IAW Department of Defense Instruction (DoDI) 1100.22, Policy and Procedures for Determining Workforce Mix.

• Reviews the Contractor performance documentation prepared by the COR on a regular basis to ensure performance is compatible with contract and mission objectives.

• The Functional Commander or civilian equivalent will develop a contingency plan if there is a reasonable doubt that the Contractor will continue to provide mission-

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 5 -essential services during a crisis IAW DoDI 1100.22, Policy and Procedures for Determining Workforce Mix, Enclosure 2, paragraph 5.b, and AFI 10-403, paragraph 1.9.1.33.1.

4.5 Quality Assurance Program Coordinator (QAPC)

• Supports the MFT in the development of the contract/order QA requirements, specifically ensuring the requirements are clearly stated and enforceable IAW AFFARS MP5346.1031.c.(2), Contracting Office Responsibilities / Roles and Responsibilities.

Provides training to the CORs and COR management (e.g., FC/FD, COR supervisor) IAW AFFARS MP5301.602-2(d)3.1, Designation, Assignment, and Responsibilities of a COR/QAPC.

• Coordinates all aspects of the QA program.

• Supports the MFT and attends MFT meetings, as requested.

• Provides training to COR Management including the COR Supervisor and FC/FD.

• Provides Phase I training to CORs at the CO’s direction.

• Maintains records of QA appointments and documents all COR and COR Management training.

4.6 Contracting Officer’s Representatives

• Self-nominates in the CORT Tool IAW Defense Federal Acquisition Regulation

Supplement (DFARS) Procedures, Guidance, and Information (PGI) 201.602-2(d)(iv), Career Development, Contracting Authority, and Responsibilities / Contracting Officers / Responsibilities, and the OUSD Memorandum, Deployment of the Department of Defense (DoD) CORT Tool, dated 21 March 2011.

• Works with MFT in conducting surveillance on Contractor performance and in evaluating Contractor performance IAW DFARS PGI 201.602-2(d)(v). As delegated:

Reports unethical conduct and instances of fraud, waste, and/or abuse to the CO IAW the OUSD Memorandum, DoD Standard for Certification of Contracting Officer's Representatives (COR) for Service Acquisitions, dated 29 March 2010. Inspects and accepts/rejects deliverables during contract/order performance and at close-out to ensure contract/order terms and conditions are met IAW the OUSD Memorandum, dated 29 March 2010. Monitors Contractor payment and verifies it is commensurate with Contractor performance IAW the OUSD Memorandum, dated 29 March 2010.

Performs inspections to evaluate and document the Contractor’s performance; notifies the CO of any significant performance deficiencies; and submits monthly inspections records to the CO.

• Documents Contractor performance via PARs, CARs, etc., IAW DFARS PGI 201.602- 2(d)(vi)(B). Issues PARs, at a minimum quarterly, and negative PARs or CARs as required. Provides reports (e.g., PARs, CARs) on Contractor performance to the CO IAW the FAR and applicable supplements and will be in compliance with all applicable guidance and clauses listed in the contract.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 6 -

• Establishes and maintains an online file containing surveillance; training documentation; initial and refresher training certificates; contract-specific training;

SOW, QASP, and Annual COR Online File Inspection Checklists IAW the FAR; and applicable supplements. The online file will be in compliance with all applicable guidance and clauses listed in the contract.

• Monitors control and disposition of Government property IAW the OUSD Memorandum, dated 29 March 2010. Participates in MFT meetings and performs/assists in accomplishing market research, gathering past performance information, evaluating technical proposals, and preparing the SOW and QASP.

• Maintains QA documentation and recommends changes/improvements to the contract, QASP, or other requirements throughout the life of the contract to provide more effective operations or eliminate unnecessary costs.

4.7 Subject Matter Experts (i.e., Legal, Finance, Small Business, and Engineering Points of Contact (POCs))

Other POCs shall assist the MFT by coordinating, reviewing, and providing input as needed and by participating as members when appointed.

4.7.1 Program Manager

Responsible for overall program. Serves as MFT lead. Prepares and submits Contractor Performance Assessment Reports (CPAR).

4.7.2 Engineer

Works with MFT in conducting market research. Responsible for providing technical and engineering support to integrated product team members as required.

4.7.3 Contractor

• May participate as a member of the MFT upon contract/order award.

• Responsible for ensuring personnel, information, system, property, facility, and international security requirements are met in accordance the Department of Defense (DD) Form 254, Department of Defense Contract Security Classification Specification, and with Government regulations, including, but not limited to, Department of Defense (DoD) 5200.2-R, Personnel Security Program; DoD 5220.22- M, National Industrial Security Program Operating Manual (NISPOM); DoD Directive (DoDD) 5205.02E; DoD Manual 5205.02-M; and DoD Operations Security (OPSEC) Program Manual.

• Complies with contractual requirements including the SOW and Contract Data Requirements Lists (CDRL). Performs services required by the contract/order, including the SOW and CDRLs. Provides the Government with contractual deliverables and other requested data by which Contractor performance will be evaluated.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 7 -

• The Contractor shall comply with inspection of services IAW the FAR and maintains an inspection process acceptable to the Government and records of inspections that shall be made available to the Government IAW the contract.

• Signs and returns a PAR within five business days. Signs and returns a negative PAR with corrective action (CA) identified by the suspense date. Signs and returns a CAR with CA plan by the suspense date.

• In accordance with the Secretary of the Air Force for Acquisition (SAF/AQ) Memorandum, Implementation of FY11 NDAA [National Defense Authorization Act] Section 8108 [of Public Law 112-10 of the Department of Defense and Full-Year Continuing Appropriations Act, 2011], Contractor Inventory, dated 13 November 2012, provides an annual count of Contractor personnel performing services if the DoD is the requiring activity, if the acquisition is using United States Government appropriated funds, and if the acquisition is equal to or greater than the Simplified Acquisition Threshold (SAT) [currently $150,000.00]. IAW the Office of the Secretary of Defense (OSD) Memorandum, Enterprise-wide Contractor Manpower Reporting Application, dated 28 November 2012, reports all Contractor labor hours, including subcontractor labor hours, required for performance of the services provided under the contract at the Enterprise-wide Contract Manpower Reporting Application (eCMRA) site: http://www.ecmra.mil

• Maintains and implements a QCP that compliments the QASP, if requested.

• Ensures that non-conforming contract services are identified and corrected and that the QCP is revised to prevent recurrence.

• Tenders to the Government for acceptance of only those services that conform to contract requirements.

• Recommends any changes to the contract that will provide more effective operations or eliminate unnecessary costs.

5 CONTRACT ASSESSMENT METHODS

5.1 Market Research

Market research will be used as a tool throughout the life of the contract to ensure the most current, efficient, and effective assessment methods and techniques of the commercial marketplace are applied to the performance of services under the contract. The results of the market research/survey of customers will be utilized in making the decision as to whether it is appropriate and in the Government’s best interests to exercise the option. These results will be documented in the CO’s Determination and Findings supporting option exercise if that course of action is selected.

5.2 Multi-Functional Team Meetings

Multi-functional team meetings will be scheduled throughout the life of the contract. Attendees at specific team meetings will vary based on the circumstances or issues to be covered at each

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 10 -

7 REVISIONS

Revisions are the responsibility of the MFT. The COR reviews the SOW and QASP annually and shall provide any proposed changes to the MFT for review. Revisions to the QASP are the joint responsibility of the MFT and are only accomplished with the approval of and by the CO.

Specific by-name membership in the MFT will be determined prior to its performance of responsibilities, envisioned to begin in Fiscal Year 2018.

8 SAFETY REQUIREMENTS

8.1 Mishap Notification

• In accordance with AFI 91-204, Safety Investigations and Reports, paragraph 2.4.6.1.1, the Contractor shall report mishaps involving damage or injury to USAF interests.

IAW AFI 91-204, paragraph 1.3.1.1, damage or injury includes damage to DoD property, occupational illness to DoD military or civilian personnel, injury to on- or off-duty DoD military personnel, injury to on-duty DoD civilian personnel, and damage to public or private property or injury or illness to non-DoD personnel caused by USAF operations.

• In accordance with AFI 91-204, paragraph 2.4.6.1.1, the Contractor shall ensure the USAF [Safety Office] and the applicable contract management authority are notified of mishaps. The Contractor shall contact the MFT (the COR, if available, or another MFT member) by telephone within one business day. The MFT will notify the Safety Office and the CO.

8.2 Mishap Investigations

In accordance with AFI 91-204, paragraph 2.4.6.1.2, the Contractor shall cooperate with USAF safety investigations.

8.3 Multi-Functional Team Surveillance

The COR will monitor Contractor reporting and review any mishap notifications. The COR will address safety and/or health issues in the PAR.

- 11 -

QUALITY ASSURANCE

SURVEILLANCE PLAN

Part 2

ASSESSMENT

MANAGEMENT

- 13 -

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 14 -

1 INTRODUCTION

These surveillance procedures have been developed to implement PBSA. They are designed to provide the COR instructions on how to perform surveillance and document findings on this contract for the APT Program.

1.1 Inspection

The COR may inspect any aspect of Contractor performance at any time during the contract, whether or not it is included in the SS. A format for documenting these inspections is found on page 31 of this plan.

1.2 Defects

Any nonconformance with contract requirements is a “defect.” The term “defective” is used in reference to a service output in the SS that does not meet the outputs associated with the standard.

1.3 Performance Responsibility

This QASP is based on the premise that the Contractor, not the Government, is responsible for the management and quality control actions required to meet the terms of the contract.

1.4 Approach

The following is the approach the team will use to assess the Contractor’s performance against the performance thresholds/metrics identified in the contract.

• The SS will summarize the performance objectives and performance thresholds required by the Government in the Contractor’s performance. These performance objectives will directly relate to mission-essential items. The performance thresholds briefly describe the minimum acceptable levels of services required for each objective and are critical to mission success.

• The COR will perform QA procedures contained herein.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 15 -

2 METHODS FOR SURVEILLANCE PERFORMANCE

2.1 Customer Complaint/Feedback

When the customer (using activity/facility) discovers unacceptable performance or a defect in service, he/she will prepare a Customer Complaint Record (CCR), identifying the discrepancy. The CCR template is included on page 27 of this document. The customer will keep one copy and forward a copy to the COR either manually or electronically. The COR will review the item and determine if the complaint is contractually valid. If valid, it is forwarded to the Contractor for CA. If a customer notifies the COR either verbally or in writing (including email) of a complaint but does not submit a CCR, the COR will generate a CCR to document the complaint. The CCR will then be submitted to the Contractor for

CA.

In addition, the COR or individual customers may from time to time wish to submit positive feedback about the Contractor’s performance under the contract. This positive feedback may be submitted on the Customer Feedback Record (CFR), found on page 26 of this plan.

In the event that the COR wants feedback from every customer/service location each month, those customers with no complaints should submit their feedback on a CFR.

Major complaints are defined as those complaints that do not meet the threshold and/or do not reasonably indicate a deficiency in the service received that would require retraining, recertification, or re-accomplishment of the identified task.

2.2 Periodic Surveillance

When periodic surveillance is used, inspections shall be performed at the frequency identified for each SS item and, otherwise, is on a “100 percent surveillance” or statistically random basis. The frequency of inspection is established (i.e., daily, weekly, monthly, or quarterly) but the specific time of occurrence need not be specified. Random and scheduled inspection will be recorded in a COR Schedule (see attachment, page 24). When the COR discovers unacceptable performance or a defect in service, he/she will generate a CAR (see attachment, page 28) identifying the discrepancy. The CAR will then be submitted to the Contractor for CA.

2.3 100 Percent Surveillance In this surveillance method, the COR inspects 100 percent of the work, through procedures designated in the SS. When the COR discovers unacceptable performance or a defect in service, he/she will generate a CAR identifying the discrepancy. The CAR will then be submitted to the Contractor for CA.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 16 -

3 INFORM CONTRACTOR’S CONTRACT MANAGER

The COR must always contact the Contractor’s manager and inform him/her of any deficiency that was observed. This may be accomplished in person or electronically, but any verbal communication must be supplemented by submission of a CCR or CAR. The Contractor will annotate on the report, provided by the COR, the CA he/she will take to correct the defect and prevent its recurrence. The Contractor will then return the completed report to the COR.

4 EVALUATE CONTRACTOR CORRECTIVE ACTION

After the Contractor documents the proposed CA, the COR will evaluate the proposed CA and forward the record, annotated with the results of the COR evaluation, to the CO for final dispensation. The CO will determine if further actions need to be taken against the Contractor, such as reductions in fee/price IAW the FAR 52.246-4, Inspection of Services, clause. The CO’s disposition of the CA and proposed CA will be annotated on the record, and copies will be returned to the COR and the Contractor.

5 REMEDIES FOR UNACCEPTABLE PERFORMANCE

In accordance with the contract Inspection of Services clause, if any of the services do not conform to contract requirements, the Government may require the Contractor to perform the services again in conformity with contract requirements at no increase in contract amount. When the defects in services cannot be corrected by re-performance, the CO may undertake the following actions:

• Remedial Action. Require the Contractor to take necessary action to ensure that future performance conforms to contract requirements.

• Payment Reduction. Reduce the contract price to reflect the reduced value of the services performed. This may include a reduction in the number of paid hours to reflect the total non-performance period.

• Penalties for Non-Performance. If the Contractor fails to promptly perform the services again or to take the necessary action to ensure future performance is in conformity with contract requirements, the Government may o Government Service Cost. By contract or otherwise, perform the services and charge to the Contractor any cost incurred by the Government that is directly related to the performance of such service.

o Invoice Adjustment. Decrease the invoice amount in relation to performance.

o Cure Notice. Issue a cure notice.

o Show Cause Notice. Issue a show cause notice.

o Terminate the contract. Terminate the contract.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 17 -

The CO will determine the action to be taken against the Contractor, such as reductions in price IAW the Inspection of Services clause. The CO’s disposition of the deficiency will be annotated, and the COR and Contractor will be notified of action.

6 CONTRACTING OFFICER’S REPRESENTATIVE MONTHLY

INSPECTION REPORT

The COR shall submit an assessment report and attach all CARs, PARs, and validated customer complaints to the CO and FC/FD no later than five working days following the month in which the actual surveillance occurred. The report shall be signed and dated by the COR and provided to the FC/FD for review prior to sending the information to the CO. Validated customer complaints and all documentation will be used to support Contractor ratings in CPARS.

7 WIDE AREA WORKFLOW

7.1 Receipt Certification

The COR will certify receipt of Contractor services via the WAWF website. Certification of services shall be accomplished on the first workday following the completion of the contract payment period to avoid payment of interest penalties.

7.2 Wide Area Workflow Training

Contracting Officer’s Representatives are responsible for obtaining, scheduling, and completing WAWF training from the Base Accounting and Finance Office.

8 REVISION TO THE QUALITY ASSURANCE SURVEILLANCE PLAN

Revisions to the QASP are the joint responsibility of the MFT. This is a living document and can be changed at any time by the CO.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 18 -

9 PERFORMANCE OBJECTIVES, PERFORMANCE THRESHOLDS,

AND METHODS OF SURVEILLANCE

9.1 SS - 1 Satisfy Total Non-Mission Capable Supply Rate

9.1.1 Statement of Work Paragraph:

3.9.3.7.6.1

9.1.2 Method(s) of Surveillance:

• 100 percent review of monthly ICS Activity Report

• Periodic surveillance

9.1.3 Performance Threshold:

As part of ICS, performance is acceptable when five percent or less of possessed time at each MOB and an Initial Operational Test and Evaluation location per month is reported as TNMCS. Performance is unacceptable when TNMCS is greater than five percent at any MOB location per month.

9.1.4 Inspection Procedure(s):

• 100 percent review of monthly ICS Data Report o If the COR validates that performance was unacceptable, the deficiency will be documented on a CAR and submitted to the Contractor’s representative.

Contractor will be requested to sign and date the CAR and state what CA will be taken to correct the deficiency and preclude reoccurrence.

o When the Contractor documents the CA, the COR will evaluate the proposed CA and, if satisfactory, will forward the record to the CO for final dispensation. The CO will, in turn, send copies of dispositioned CAR to the COR and the Contractor for their records.

o When a dispositioned CAR has been received, one copy of the completed CAR will be retained in the QA file maintained by the COR.

• Periodic surveillance o See Pg. 15, Paragraph 2.2, “Periodic Surveillance.”

o See Pg. 16, Paragraph 4, “Evaluate Contractor Corrective Action.”

o See Pg. 16, Paragraph 5, “Remedies For Unacceptable Performance.”

9.2 SS - 2 Issue Effectiveness Rate

9.2.1 Statement of Work Paragraph:

3.9.3.7.6.2

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 19 -

9.2.2 Method(s) of Surveillance:

• 100 percent review of monthly ICS Data Report

• Periodic surveillance

9.2.3 Performance Threshold:

As part of ICS, performance is acceptable when the Issue Effectiveness rate at each MOB per month is reported as greater than 85 percent for reparable(s) and greater than 80 percent for any consumables. Performance is unacceptable when the Issue Effectiveness rate is less than 85 percent for reparable(s) and less than 80 percent for consumables at any MOB. Non-MICAP backorders should be filled within 72 hours 90 percent of the time, and MICAP backorders should be filled within 24 hours.

9.2.4 Inspection Procedure(s):

• 100 Percent Review of Monthly ICS Data Report o See Pg. 18, Paragraph 9.1.4, “100 percent review of monthly ICS Data Report.”

o See Pg. 15, Paragraph 2.2, “Periodic Surveillance.”

o See Pg. 16, Paragraph 4, “Evaluate Contractor Corrective Action.”

o See Pg. 16, Paragraph 5, “Remedies For Unacceptable Performance.”

• Periodic Surveillance o See Pg. 15, Paragraph 2.2, “Periodic Surveillance.”

o See Pg. 16, Paragraph 4, “Evaluate Contractor Corrective Action.”

o See Pg. 16, Paragraph 5, “Remedies For Unacceptable Performance.”

9.3 SS – 3 Aircrew Training Devices Certified for Training

9.3.1 Statement of Work Paragraph

3.5.3.2.3.2

9.3.2 Method of Surveillance:

Recurring SIMCERTs

9.3.3 Frequency:

• Annual certification of each GBTS ATD

• Periodic surveillance

9.3.4 Performance Threshold:

The Contractor shall maintain all ATDs, such that they are fully certified for training.

9.3.5 Inspection Procedure(s):

• SIMCERT support

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 20 -

The Contractor shall support the Air Force-conducted SIMCERT of the ATDs IAW AFI 36-2251, Chapter 6.

o See Pg. 15, Paragraph 2.2, “Periodic Surveillance.”

o See Pg. 16, Paragraph 4, “Evaluate Contractor Corrective Action.”

o See Pg. 16, Paragraph 5, “Remedies For Unacceptable Performance.”

• Aircrew Training Device Performance o The Contractor shall document the performance of the ATDs in Qualification Test

Guides (QTG). The Contractor shall make required QTG flight performance data available prior to each visit to allow review of training device performance.

o The Contractor shall document all identified degradations/deficiencies IAW SOW section 3.5.3.2.3.2, “SIMCERT.”

o At his/her discretion, the COR may periodically survey SIMCERT activities.

o If the COR validates that performance was unacceptable, the deficiency will be documented on a CAR and submitted to the Contractor’s representative. The Contractor will be requested to sign and date the CAR and state what CA will be taken to correct the deficiency and preclude reoccurrence.

o When the Contractor documents the CA, the COR will evaluate the proposed CA and, if satisfactory, will forward the record to the CO for final disposition. The CO will, in turn, send copies of dispositioned CAR to the COR and the Contractor for their records.

o When a dispositioned CAR has been received, one copy of the completed CAR will be retained in the QA file maintained by the COR.

o If correction of the deficiency requires re-performance of the service, the COR shall perform re-inspections to determine if the Contractor adequately re-performed and service is acceptable.

9.4 SS – 4 Trainer Availability

9.4.1 Statement of Work Paragraph

3.5.3.2.5.2

9.4.2 Method(s) of Surveillance:

• Periodic surveillance

• Customer complaint

9.4.3 Frequency:

• Daily

• As required

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 21 -

9.4.4 Performance Threshold:

Maintain training device availability consistent with the APT GBTS System Specification, section 3.5.10, “Reliability and Maintainability.” Operational Availability (AO) shall be at least 95 percent for the ATDs. AO shall be at least 98 percent for all other GBTS devices, including, but not limited to, the aircrew ground egress trainer, ejection seat trainer, and part-task trainers.

9.4.5 Inspection Procedure(s):

• 100 percent review of Contractor Device Performance Report o If the COR validates that performance was unacceptable, the deficiency will be documented on a CAR and submitted to the Contractor’s representative. The Contractor will be requested to sign and date the CAR and state what CA will be taken to correct the deficiency and preclude reoccurrence.

o When the Contractor documents the CA, the COR will evaluate the proposed CA and, if satisfactory, will forward the record to the CO for final dispensation. The CO will, in turn, send copies of the dispositioned CAR to the COR and the Contractor for their records.

o When a dispositioned CAR has been received, one copy of the completed CAR will be retained in the QA file maintained by the COR.

• Customer complaint o See Pg. 15, Paragraph 2.1, “Customer Complaint/Feedback.”

o See Pg. 15, Paragraph 2.2, “Periodic Surveillance.”

o See Pg. 16, Paragraph 4, “Evaluate Contractor Corrective Action.”

o See Pg. 16, Paragraph 5, “Remedies for Unacceptable Performance.”

9.5 SS – 5 Provide Accurate Resolutions to Customer Needs and Agency Requirements

9.5.1 Statement of Work Paragraph

3.5.3.2

9.5.2 Method of Surveillance:

Periodic surveillance

9.5.3 Performance Threshold:

Provide at least 90 percent acceptable responses to customer needs and agency requirements during each month.

9.5.4 Inspection Procedure(s):

Periodic surveillance

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 22 -o See Pg. 15, Paragraph 2.2, “Periodic Surveillance.”

o See Pg. 16, Paragraph 4, “Evaluate Contractor Corrective Action.”

o See Pg. 16, Paragraph 5, “Remedies For Unacceptable Performance.”

9.6 SS – 6 Provide Organizational-Level Field Service Representative as part of Sustainment

9.6.1 Statement of Work Paragraph

3.9.3.8

9.6.2 Method of Surveillance:

• 100 percent review of monthly organizational-level (O-Level) Contract Field Service

Report

• Periodic surveillance

9.6.3 Performance Threshold:

The Contractor shall provide O-Level FSR personnel and subject matter expertise to sufficiently support maintenance operations with technical advisory, supplemental training, and reach-back assistance to support aircraft availability.

9.6.4 Inspection Procedures:

• 100 percent review of monthly O-Level Contract Field Service Report o If the COR validates that performance was unacceptable, the deficiency will be documented on a CAR and submitted to the Contractor’s representative. The Contractor will be requested to sign and date the CAR and state what CA will be taken to correct the deficiency and preclude reoccurrence.

o When the Contractor documents the CA, the COR will evaluate the proposed CA and, if satisfactory, will forward the record to the CO for final dispensation. The CO will, in turn, send copies of dispositioned CAR to the COR and the Contractor for their records.

o When a dispositioned CAR has been received, one copy of the completed CAR will be retained in the QA file maintained by the COR.

• Periodic surveillance o See Pg. 15, Paragraph 2.2, “Periodic Surveillance.”

o See Pg. 16, Paragraph 4, “Evaluate Contractor Corrective Action.”

o See Pg. 16, Paragraph 5, “Remedies For Unacceptable Performance.”

- 23 -

MULTI-FUNCTIONAL TEAM MEMBERSHIP

Proposal Number FA8617-17-R-6219 Type of Service: _________________________________

FC/FD: ___________________ Date: __________________

Chief of Contracting**: __________________ Date: __________________

Contracting Officer: __________________ Date: __________________

Primary COR: __________________ Date: __________________

COR Supervisor: __________________ Date: __________________

Alternate COR: __________________ Date: __________________

QAPC: ___________________ Date: __________________

Program Manager**: ___________________ Date: __________________

Engineer**: ___________________ Date: __________________

Subject Matter Expert**: ____________________ Date: __________________

MFT Contractor*: ____________________ Date: __________________

Designated Official**: ____________________ Date: __________________

Accepted By: ____________________ Date: __________________ Contracting Officer

* Optional - Contractor (signs after award) will have limited participation on the MFT so as to not be involved in any inherently governmental responsibilities.

** As applicable

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 25 -

COR SURVEILLANCE CHECKLIST

A = Acceptable U=Unacceptable N/A= Not applicable

METHOD OF

ASSESSMENT-

CONTRACTOR'S

NAME:

CONTRACT

NUMBER:

LOCATION:

Building # WPAFB

A U N/A DATE TIME

QAP

SIGNATURE

SS-1 -

Paragraph

SS-2

SS-3

SS-4

DATE:

Functional Director/Commander Date

EXAMPLE OF COR SURVEILLANCE CHECKLIST

- 26 -

CUSTOMER FEEDBACK RECORD

DATE/TIME OF EVENT

DETAILS OF EVENT

ORGANIZATION

BUILDING NUMBER

INDIVIDUAL

PHONE

NUMBER

NATURE OF COMMENT

CONTRACT REFERENCE

VALIDATION

DATE/TIME CONTRACTOR INFORMED OF OBSERVATION

OTHER COMMENTS OR NOTES

RECEIVED BY (NAME & DATE)

EXAMPLE OF CUSTOMER FEEDBACK RECORD

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 27 -

Customer Complaint Record Template, Feb 05 (SAF/AQCP) One copy each for Complainant, COR, Contractor, and Contracting Officer

CUSTOMER COMPLAINT RECORD

DATE/TIME OF

COMPLAINT

SOURCE OF COMPLAINT

ORGANIZATION

BUILDING NUMBER

INDIVIDUAL

PHONE

NATURE OF COMPLAINT

CONTRACT REFERENCE

VALIDATION

DATE/TIME CONTRACTOR INFORMED OF COMPLAINT

ACTION TAKEN BY CONTRACTOR

RECEIVED/VALIDATED BY

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 28 -

CAR Template, Feb 05 (SAF/AQCP)

CORRECTIVE ACTION REPORT

(If more space is needed, use reverse and identify by number)

1. CONTRACTOR

2. CONTRACT NUMBER

3. TYPE OF SERVICES

4. FUNCTIONAL AREA

5. SUSPENSE DATE

6.

CONTROL

7. DEFICIENCY MAJOR MINOR

FINDING:

FINDING IMPACT:

Please respond with a written corrective action plan that details the corrective action of the cited deficiency, the cause of the deficiency, and actions taken to prevent recurrence by Suspense Date in Block 5. If date was not entered in Block 5, the Contractor is not required to provide a response.

8. CONTRACTING OFFICER REPRESENTATIVE (COR)

TYPED NAME AND GRADE

SIGNATURE AND DATE

9. ISSUING AUTHORITY

TYPED NAME AND GRADE

SIGNATURE AND DATE

10. COR RESPONSE TO CONTRACTOR CORRECTIVE ACTION AND ACTION TAKEN TO PREVENT RECURRENCE

11. COR DETERMINATION

ACCEPTED REJECTED

12. CLOSE DATE

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 29 -

CORRECTIVE ACTION REPORT

Continuation Sheet

(Number to correspond with applicable Item Number on reverse)

CAR Template, Feb 05 (SAF/AQCP)

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 30 -

CORRECTIVE ACTION REPORT (CAR) INSTRUCTIONS

Block 1. Enter Contractor Name.

Block 2. Enter Contract Number.

Block 3. Enter Contract Name or Type of Services.

Block 4. Enter Functional Area of the Contract.

Block 5. Enter the assigned suspense date given to the Contractor to provide a response to the CAR. A date must be entered for a Major CAR. A date is optional at the discretion of the Contracting Officer’s Representative (COR) initiating the CAR if the finding is Minor.

Block 6. All CARs will be tracked with a control number. The functional commander/functional director is the Issuing Authority for his/her contract quality assurance evaluators generating CARs. By providing the control number to the COR originating the CAR, the Issuing Authority demonstrates concurrence with the finding. The control number will be made up of the first two letters of the name of the site or contract, the last two digits of the calendar year, and a three-digit number starting with 001 and progressing upward throughout the calendar year (e.g., WP04-001).

Block 7.

1. Check the block that indicates whether the identified deficiency is assigned as a Major or Minor finding. See Quality Assurance Surveillance Plan, paragraph 6.1.4, for the definitions of Major and Minor findings and associated explanations.

2. Finding: Clearly state the details of the finding, followed by a reference to the stated contractual requirement. The reference must state the portion of the contract, part, section, paragraph, and subparagraph and must make a complete, brief quotation of the contract reference.

3. State the impact that the finding has or could have on the accomplishment of the mission that the contract provides.

4. When determining how long to give the Contractor to respond to the identified finding, the normal is 10 working days. The suspense date should reflect this unless the finding requires a greater amount of time to come to solution. Major findings require a Contractor response. Minor findings may require a Contractor response at the discretion of the COR initiating the CAR and/or the Issuing Authority.

Blocks 8 and 9. Contract CORs initiating a CAR must sign in Block 8, and the Issuing Authority signs in Block 9. Each annotates the date at the time of signature.

Block 10. Upon review of the Contractor’s reply, the originator of the CAR will enter comments regarding acceptance or rejection of the Contractor’s response. This block may also contain any comments regarding follow-on inspections conducted or needed at a later date to validate that the finding has not reoccurred.

Block 11. The COR selects Accept or Reject after reviewing the Contractor’s response.

Block 12. The COR enters a close date after advising the Issuing Authority that the Contractor’s response is acceptable and the Issuing Authority concurs. The Issuing Authority will then forward the closed CAR to the Contractor and report accordingly in the monthly Certificate of Service or other applicable contracting officer services report. This routing remains the same if the Contractor response is unacceptable, and the CAR remains open until an acceptable response is received, except that the Issuing Authority forwards the open CAR back to the Contractor for additional information.

Continuation Sheet. The Continuation Sheet is to be used to expand on any information that will not fit in the applicable numbered block.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 31 -

NON-SERVICES SUMMARY (SS) INSPECTION FORM

This form addresses requirements in the contract that should be periodically inspected but are not addressed in the SS. The Contracting Officer’s Representative (COR) can use this form as is or change the frequency and/or number of items to be inspected. Any deficiencies noted shall be reported to the Contractor using a contract deficiency record.

Date

Non-SS Items Checked (Description and SOW Paragraph number) Acceptable Unacceptable COR Initials

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 32 -

ACRONYM LIST

AFFARS Air Force Federal Acquisition Regulation Supplement

AFI Air Force Instruction

AFLCMC Air Force Life Cycle Management Center

AO Operational Availability

APT Advanced Pilot Training

ATD Aircrew Training Devices

CA Corrective Action

CAR Corrective Action Report

CCR Customer Complaint Record

CDRL Contract Data Requirements List

CFR Customer Feedback Record

CO Contracting Officer

COR Contracting Officer’s Representative

CORT Contracting Officer Representative Tracking (Tool)

CPAR Contractor Performance Assessment Report

CPARS Contractor Performance Assessment Reporting System

DD Department of Defense

DFARS Defense Federal Acquisition Regulation Supplement

DoD Department of Defense

DoDD Department of Defense Directive

DoDI Department of Defense Instruction

FAR Federal Acquisition Regulation

FC/FD Functional Commander / Functional Director

FSR Field Service Representative

GBTS Ground Based Training System

IAW In Accordance With

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 33 -

ICS Interim Contractor Support

MFT Multi-Functional Team

MICAP Mission Impaired Capable Awaiting Parts

MOB Main Operating Base

O-Level Organizational-Level

OSD Office of the Secretary of Defense

OUSD Office of the Under Secretary of Defense

PAR Performance Assessment Report

PBSA Performance-Based Service Acquisition

POC Point of Contact

QA Quality Assurance

QAPC Quality Assurance Program Coordinator

QASP Quality Assurance Surveillance Plan

QCP Quality Control Plan

QTG Qualification Test Guides

SAF/AQCP Deputy Assistant Secretary of the Air Force for Contracting (Policy Division)

SIMCERT Simulator Certification

SOW Statement of Work

SS Services Summary

TNMCS Total Non-Mission Capable Supply

USAF United States Air Force

WAWF Wide Area Workflow