AMEND_0001_-_Section_J_-_Attachment_16_-_Quality_Assurance_Surveillance_Plan_for_the_Advanced_Pilot_Training_Program.pdf

Solicitation FA8617-17-R-6219 AMEND 0001 Section J – Attachment 16

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

QUALITY ASSURANCE SURVEILLANCE PLAN

for the

ADVANCED PILOT TRAINING PROGRAM

FA8617-17-R-6219 AMEND 0001

3 February 2017

Version 1.0

Prepared by

Air Force Life Cycle Management Center (AFLCMC) Training Aircraft Division (WLZ)

Advanced Pilot Training Branch (WLZG)

- iii -

PART I: PERFORMANCE MANAGEMENT

1 INTRODUCTION

2 OBJECTIVE

3 GOALS OF THE MULTI-FUNCTIONAL TEAM

4 ROLES AND RESPONSIBILITIES

4.1 Chief of the Contracting Office

4.2 Contracting Officer (CO)

4.3 Contracting Officer’s Representative Supervisor

4.4 Functional Commander / Functional Director (FC/FD)

4.5 Quality Assurance Program Coordinator (QAPC)

4.6 Contracting Officer’s Representatives

4.7 Subject Matter Experts (i.e., Legal, Finance, Small Business, and Engineering Points of Contact (POCs))

5 CONTRACT ASSESSMENT METHODS

5.1 Market Research

5.2 Multi-Functional Team Meetings

5.3 Data Review and Corrective Actions

6 PERFORMANCE ASSESSMENT

6.1 Contractor Responsibility

6.2 Approach

6.3 Services Summary (SS)

6.4 Quality Assurance Procedures

6.5 Contractor Performance Assessment Reporting System

6.6 Combating Trafficking in Persons

7 REVISIONS

8 SAFETY REQUIREMENTS

8.1 Mishap Notification

8.2 Mishap Investigations

8.3 Multi-Functional Team Surveillance

PART II: ASSESSMENT MANAGEMENT

1 INTRODUCTION

1.1 Inspection

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- iv -

1.2 Defects

1.3 Performance Responsibility

1.4 Approach

2 METHODS FOR SURVEILLANCE PERFORMANCE

2.1 Customer Complaint/Feedback

2.2 Periodic Surveillance

2.3 100 Percent Surveillance

3 INFORM CONTRACTOR’S CONTRACT MANAGER

4 EVALUATE CONTRACTOR CORRECTIVE ACTION

5 REMEDIES FOR UNACCEPTABLE PERFORMANCE

6 CONTRACTING OFFICER’S REPRESENTATIVE MONTHLY

INSPECTION REPORT

7 WIDE AREA WORKFLOW

7.1 Receipt Certification

7.2 Wide Area Workflow Training

8 REVISION TO THE QUALITY ASSURANCE SURVEILLANCE PLAN

9 PERFORMANCE OBJECTIVES, PERFORMANCE THRESHOLDS,

AND METHODS OF SURVEILLANCE

9.1 SS - 1 Satisfy Total Non-Mission Capable Supply Rate

9.2 SS - 2 Issue Effectiveness Rate

9.3 SS – 3 Aircrew Training Devices Certified for Training

9.4 SS – 4 Trainer Availability

9.5 SS – 5 Provide Accurate Resolutions to Customer Needs and Agency Requirements.. 21

9.6 SS – 6 Provide Organizational-Level Field Service Representative as part of Sustainment

MULTI-FUNCTIONAL TEAM MEMBERSHIP

CONTRACTING OFFICER REPRESENTATIVE SCHEDULE

COR SURVEILLANCE CHECKLIST

CUSTOMER FEEDBACK RECORD

CUSTOMER COMPLAINT RECORD

CORRECTIVE ACTION REPORT

NON-SERVICES SUMMARY (SS) INSPECTION FORM

ACRONYM LIST

- v -

QUALITY ASSURANCE

SURVEILLANCE PLAN

Part 1

PERFORMANCE

MANAGEMENT

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 1 -

1 INTRODUCTION

This Quality Assurance Surveillance Plan (QASP) has been developed to implement Air Force Instruction (AFI) 63-101/20-101, Chapter 8, Acquisition of Services of Performance-Based Service Acquisitions (PBSA), for the Advanced Pilot Training (APT) Program aircraft and Ground Based Training System (GBTS) procurement. The QASP is designed to assess contract success and to assess and document Contractor performance in accordance with (IAW) the requirements set forth in the APT contract for the production and sustainment phases. The specifics of this work performed on this contract are defined in the APT Statement of Work

(SOW).

The QASP also provides the Contracting Officer’s Representative (COR) an effective and systematic surveillance method for each listed service. In addition, this QASP implements the multi-functional team (MFT) concept. The goal of the MFT is to give all members/stakeholders the ability to maintain the highest quality-representative service to the user and propose/initiate improvements throughout the life of the contract. Under this concept, continual contract assessment will be focused on ensuring that the contract, as written, continues to meet the needs of the user. The signatures, on the attached MFT Membership List charter signature page, indicate membership in the MFT. Changes to team members will require an amendment to the charter signature page.

The QASP is issued with the solicitation for informational purposes only. The QASP procedures can be changed unilaterally by the Government at any time.

In an effort to streamline the quality assurance (QA) provisions described in AFI 63-101, Integrated Life Cycle Management, this QASP has been developed to cover all applicable APT contracts with service-designated contract line item numbers.

2 OBJECTIVE

The APT Program will replace the T-38C used in the United States Air Force’s (USAF) Specialized Undergraduate Pilot Training advanced phase fighter/bomber track and in the Introduction to Fighter Fundamentals course. The APT Program will provide student pilots with the foundational flying skills and core competencies required to transition into current generation fighter/bomber aircraft.

The objective of this plan is to identify methods used for evaluation and surveillance of the quality of all applicable services provided by the Contractor throughout all production and sustainment activities, including Interim Contractor Support (ICS). Contractor logistics support for the APT GBTS, as identified in the APT SOW, is also included, but not limited to, Training System Support Center support (including configuration management, logistics support, Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 2 -integration, software modification, and validation of all GBTS Software) and the simulator maintenance activities for the operations, maintenance, and support for the APT GBTS.

The QASP details the USAF’s plan for monitoring the Contractor’s performance while providing oversight capability to make certain the Contractor is delivering products and services that conform to contract requirements. In addition, the QASP’s goal is to continually explore and implement accurate performance measurements reflecting the continual improvements needed to maximize customer service and result in lower costs, faster service, and higher quality. The COR will use a variety of surveillance and assessment methods to evaluate the Contractor’s performance. This QASP is not intended to duplicate the Contractor’s Quality Control Plan (QCP). This plan is a living document.

3 GOALS OF THE MULTI-FUNCTIONAL TEAM

The MFT should strive to achieve efficiencies in the contract performance and to achieve improved customer support. The team is composed of stakeholders in the acquisition process and is responsible for the acquisition throughout the life of the contract. The goal of the team is to define the requirement in performance-based terms, ensure Contractor compliance with contract requirements, and ensure that the contract, as written, continues to meet the needs of the functional organization. The team promotes good business decisions to meet customer requirements, fosters partnerships with industry to ensure exchanges of information with the Contractor and other business experts, develops a performance-based acquisition strategy to include performance metrics, develops PBSA and technical requirements, and develops the independent Government cost estimate. The team is also responsible for managing contract performance IAW the QASP, assessing and documenting Contractor performance IAW the QASP, identifying opportunities to improve performance throughout the life of the contract, and completing and reporting annual performance reviews in the Contractor Performance Assessment Reporting System (CPARS), as required. In addition, another goal is implementation and execution of the acquisition strategy to ensure support of the mission. The goal entails planning, programming, and budgeting for adequate funds, ensuring program execution within the approved funding, developing the milestones and managing their execution to ensure the contract supports mission requirements within the approved scheduled baseline, ensuring the program maintains accountability, and planning and management of contract services. After contract award, the APT Contractor may be invited to become a limited-participation member of the MFT (i.e., not involved in any inherently governmental responsibilities).

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 3 -

4 ROLES AND RESPONSIBILITIES

4.1 Chief of the Contracting Office

• Serves as Business Advisor to senior leadership.

• Updates the Program Manager, Program Executive Officer, AFLCMC Commander, and other headquarters directors annually on the health of the service acquisitions.

• Ensures personnel on the MFT receive the appropriate level of training to meet local needs.

• Ensures compliance with the Federal Acquisition Regulation (FAR) and supplements.

• Ensures functional area leadership receives training.

4.2 Contracting Officer (CO)

• Ensures acquisition processes (i.e., market research, past performance information gathering, source selection, solicitations issuance, proposal evaluation, and contract award) are aggressively performed.

• Participates in MFT meetings.

• Delegates authority for inspection and/or acceptance to the COR IAW the terms and conditions of the contract. Informs the Contractor of the names, duties, and limitations of authority for all CORs assigned to the contract.

• Advises the MFT on QASP development including incentives and remedies tied to performance objectives and performance thresholds IAW Air Force Federal Acquisition Regulation Supplement (AFFARS) MP5346.103-1.b.(1). Develops and awards a contract/order based on MFT requirements and acquisition strategy. Only individual authorized to enter into or change the contract for supplies and/or services by obligating the Government.

• Determines if a COR is required to conduct/document surveillance of Contractor performance IAW AFFARS MP5301.602-2(d)1.2, Designation, Assignment, and Responsibilities of a Contracting Officer’s Representative (COR), Revised 3 June 2016.

Appoints and approves COR nominations in the Contracting Officer Representative Tracking (CORT) Tool and submits the Designation of COR Memorandum to the COR and Contractor for signature, thereby ensuring the Contractor is notified of the COR assigned to the contract/order and his/her areas of responsibility and limitation of authority IAW Defense Acquisition Regulation Supplement (DFARS) PGI 201.602- 2(d)(v)(A) and AFFARS MP5301.602-2(d)1.5. Upon transfer/termination of the COR, issues a Revocation (Termination) of Designation of COR Memorandum IAW AFFARS MP5301.602-2(d)1.10-1.12. Uploads Designation of COR Memorandum and, if applicable, Termination/Revocation of Contracting Officer’s Representative Designation Memorandum in the CORT Tool IAW DFARS PGI 201.602-2(d)(v)(C), Responsibilities, and the Office of the Under Secretary of Defense (OUSD) Memorandum, dated 21 March 2011. Provides contract-specific COR training to the COR IAW AFFARS MP5301.602-2(d)1.4.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 4 -

• Reviews COR reports (e.g., Performance Assessment Reports (PAR), Corrective Action Reports (CAR)) on Contractor performance IAW AFFARS MP5301.602-2(d)1.6.

Inspects COR Online File and COR Profile in the CORT Tool, at a minimum, annually, IAW DFARS PGI 201.602-2(d)(vii). Reviews and signs the Annual COR Online File Inspection Checklist and uploads checklist in the CORT Tool IAW DFARS PGI 201.602-2(d)(vii).

• Responsible for administration of the contract/order except as delegated to the Administrative Contracting Officer. Takes action, if necessary, to enforce contractual requirements.

4.3 Contracting Officer’s Representative Supervisor

• Nominates a primary and alternate COR unless another QA organization or method is used to satisfy surveillance requirements (e.g., centralized performance management office).

• Ensures nomination package specifies nominee’s qualifications.

• Upon notification of the COR nomination in the CORT Tool (within Wide Area Workflow (WAWF) E-Business Suite), approves COR nomination in the CORT Tool IAW DFARS PGI 201.602-2(d)(iv) and the OUSD Memorandum, dated 21 March 2011. Inspects the COR Online File and the COR Profile in the CORT Tool, at a minimum, annually, IAW DFARS PGI 201.602-2(d)(vii). Reviews and signs the Annual COR Online File Inspection Checklist IAW DFARS PGI 201.602-2(d)(vii).

• Ensures the COR is afforded the necessary resources (i.e., time, supplies, equipment, opportunity) to perform the designated functions.

• Reviews the Contractor performance documentation prepared by the COR on a regular basis to ensure performance is compatible with contract and mission objectives.

• Evaluates the COR on performance of his/her COR duties – both pre- and post-award – as part of his/her performance assessment throughout the period of the contract.

• Solicits input on the COR’s performance of COR duties from the CO.

4.4 Functional Commander / Functional Director (FC/FD)

• Ensures requirements documents developed by the MFT satisfy mission requirements, are performance based, foster innovation, and define metrics. Metrics should provide meaningful measurements of the Contractor progress aimed at attaining desired outcomes.

• Identifies mission-essential services and develops the necessary documents IAW Department of Defense Instruction (DoDI) 1100.22, Policy and Procedures for Determining Workforce Mix.

• Reviews the Contractor performance documentation prepared by the COR on a regular basis to ensure performance is compatible with contract and mission objectives.

• The Functional Commander or civilian equivalent will develop a contingency plan if there is a reasonable doubt that the Contractor will continue to provide mission-

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 5 -essential services during a crisis IAW DoDI 1100.22, Policy and Procedures for Determining Workforce Mix, Enclosure 2, paragraph 5.b, and AFI 10-403, paragraph 1.9.1.33.1.

4.5 Quality Assurance Program Coordinator (QAPC)

• Supports the MFT in the development of the contract/order QA requirements, specifically ensuring the requirements are clearly stated and enforceable IAW AFFARS MP5346.1031.c.(2), Contracting Office Responsibilities / Roles and Responsibilities.

Provides training to the CORs and COR management (e.g., FC/FD, COR supervisor) IAW AFFARS MP5301.602-2(d)3.1, Designation, Assignment, and Responsibilities of a COR/QAPC.

• Coordinates all aspects of the QA program.

• Supports the MFT and attends MFT meetings, as requested.

• Provides training to COR Management including the COR Supervisor and FC/FD.

• Provides Phase I training to CORs at the CO’s direction.

• Maintains records of QA appointments and documents all COR and COR Management training.

4.6 Contracting Officer’s Representatives

• Self-nominates in the CORT Tool IAW Defense Federal Acquisition Regulation

Supplement (DFARS) Procedures, Guidance, and Information (PGI) 201.602-2(d)(iv), Career Development, Contracting Authority, and Responsibilities / Contracting Officers / Responsibilities, and the OUSD Memorandum, Deployment of the Department of Defense (DoD) CORT Tool, dated 21 March 2011.

• Works with MFT in conducting surveillance on Contractor performance and in evaluating Contractor performance IAW DFARS PGI 201.602-2(d)(v). As delegated:

Reports unethical conduct and instances of fraud, waste, and/or abuse to the CO IAW the OUSD Memorandum, DoD Standard for Certification of Contracting Officer's Representatives (COR) for Service Acquisitions, dated 29 March 2010. Inspects and accepts/rejects deliverables during contract/order performance and at close-out to ensure contract/order terms and conditions are met IAW the OUSD Memorandum, dated 29 March 2010. Monitors Contractor payment and verifies it is commensurate with Contractor performance IAW the OUSD Memorandum, dated 29 March 2010.

Performs inspections to evaluate and document the Contractor’s performance; notifies the CO of any significant performance deficiencies; and submits monthly inspections records to the CO.

• Documents Contractor performance via PARs, CARs, etc., IAW DFARS PGI 201.602- 2(d)(vi)(B). Issues PARs, at a minimum quarterly, and negative PARs or CARs as required. Provides reports (e.g., PARs, CARs) on Contractor performance to the CO IAW the FAR and applicable supplements and will be in compliance with all applicable guidance and clauses listed in the contract.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 6 -

• Establishes and maintains an online file containing surveillance; training documentation; initial and refresher training certificates; contract-specific training;

SOW, QASP, and Annual COR Online File Inspection Checklists IAW the FAR; and applicable supplements. The online file will be in compliance with all applicable guidance and clauses listed in the contract.

• Monitors control and disposition of Government property IAW the OUSD Memorandum, dated 29 March 2010. Participates in MFT meetings and performs/assists in accomplishing market research, gathering past performance information, evaluating technical proposals, and preparing the SOW and QASP.

• Maintains QA documentation and recommends changes/improvements to the contract, QASP, or other requirements throughout the life of the contract to provide more effective operations or eliminate unnecessary costs.

4.7 Subject Matter Experts (i.e., Legal, Finance, Small Business, and Engineering Points of Contact (POCs))

Other POCs shall assist the MFT by coordinating, reviewing, and providing input as needed and by participating as members when appointed.

4.7.1 Program Manager

Responsible for overall program. Serves as MFT lead. Prepares and submits Contractor Performance Assessment Reports (CPAR).

4.7.2 Engineer

Works with MFT in conducting market research. Responsible for providing technical and engineering support to integrated product team members as required.

4.7.3 Contractor

• May participate as a member of the MFT upon contract/order award.

• Responsible for ensuring personnel, information, system, property, facility, and international security requirements are met in accordance the Department of Defense (DD) Form 254, Department of Defense Contract Security Classification Specification, and with Government regulations, including, but not limited to, Department of Defense (DoD) 5200.2-R, Personnel Security Program; DoD 5220.22- M, National Industrial Security Program Operating Manual (NISPOM); DoD Directive (DoDD) 5205.02E; DoD Manual 5205.02-M; and DoD Operations Security (OPSEC) Program Manual.

• Complies with contractual requirements including the SOW and Contract Data Requirements Lists (CDRL). Performs services required by the contract/order, including the SOW and CDRLs. Provides the Government with contractual deliverables and other requested data by which Contractor performance will be evaluated.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 7 -

• The Contractor shall comply with inspection of services IAW the FAR and maintains an inspection process acceptable to the Government and records of inspections that shall be made available to the Government IAW the contract.

• Signs and returns a PAR within five business days. Signs and returns a negative PAR with corrective action (CA) identified by the suspense date. Signs and returns a CAR with CA plan by the suspense date.

• In accordance with the Secretary of the Air Force for Acquisition (SAF/AQ) Memorandum, Implementation of FY11 NDAA [National Defense Authorization Act] Section 8108 [of Public Law 112-10 of the Department of Defense and Full-Year Continuing Appropriations Act, 2011], Contractor Inventory, dated 13 November 2012, provides an annual count of Contractor personnel performing services if the DoD is the requiring activity, if the acquisition is using United States Government appropriated funds, and if the acquisition is equal to or greater than the Simplified Acquisition Threshold (SAT) [currently $150,000.00]. IAW the Office of the Secretary of Defense (OSD) Memorandum, Enterprise-wide Contractor Manpower Reporting Application, dated 28 November 2012, reports all Contractor labor hours, including subcontractor labor hours, required for performance of the services provided under the contract at the Enterprise-wide Contract Manpower Reporting Application (eCMRA) site: http://www.ecmra.mil

• Maintains and implements a QCP that compliments the QASP, if requested.

• Ensures that non-conforming contract services are identified and corrected and that the QCP is revised to prevent recurrence.

• Tenders to the Government for acceptance of only those services that conform to contract requirements.

• Recommends any changes to the contract that will provide more effective operations or eliminate unnecessary costs.

5 CONTRACT ASSESSMENT METHODS

5.1 Market Research

Market research will be used as a tool throughout the life of the contract to ensure the most current, efficient, and effective assessment methods and techniques of the commercial marketplace are applied to the performance of services under the contract.

5.2 Multi-Functional Team Meetings

Multi-functional team meetings will be scheduled throughout the life of the contract. Attendees at specific team meetings will vary based on the circumstances or issues to be covered at each meeting. Continual contract assessment will be focused on ensuring that the contract, as written, continues to meet the needs of the customer. Team members will consider whether the contract includes tasks that are no longer required, whether there are needs related to the contract’s purpose that are not covered by the contract, and whether any customer complaints regarding

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 8 -contract scope have been received. The success of the contract is dependent upon a combined effort of all members of the MFT.

5.3 Data Review and Corrective Actions

Contracting Officer’s Representative inspection records will be collected monthly and reviewed for acceptable and unacceptable inspections. All performance assessment data will be reviewed.

If performance improvement is necessary, the CA will be requested from the Contractor.

6 PERFORMANCE ASSESSMENT

6.1 Contractor Responsibility

This QASP is based on the premise that the Contractor, not the Government, is responsible for the management and quality control actions needed to meet the standards of the contract.

6.2 Approach

The following is the approach the team will use to assess the Contractor’s performance against the performance thresholds/metrics identified in the contract.

6.3 Services Summary (SS)

An SS will summarize the performance objectives and performance thresholds required by the Government in the Contractor’s performance. These performance objectives will directly relate to mission-essential items. The performance thresholds briefly describe the minimum acceptable levels of service required for each objective and are critical to mission success. The Contractor shall be aware that the absence of any contract requirement in the APT Systems Specifications does not detract from its enforceability nor limit the rights or remedies of the Government under any other provision from the contract.

6.4 Quality Assurance Procedures

The COR will perform QA procedures IAW the QASP, Part 2, “Assessment Management.”

6.4.1 Assessment Values

Green No significant issues; issues may be resolved by normal Contractor management.

Yellow Issue(s) but Contractor has an adequate mitigation or corrective action plan in place.

Red Issue(s) with inadequate or no Contractor mitigation or corrective action plan.

Any “red” assessment shall include the Government proposed actions with respect to the negative assessment.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 9 -

6.5 Contractor Performance Assessment Reporting System

• The total estimated value of the contract equals or exceeds one million dollars

($1,000,000.00); therefore, CPARS reporting is required.

• In accordance with FAR Part 42, Contract Administration and Audit Services, the MFT will evaluate Contractor performance. CPARS will be updated documenting Contractor performance. The Assessing Official Representative will be the COR, and the reviewing official will be the FC/FD. The Program Manager will, at a minimum, review the PARs and other information contained in the COR Online File to prepare and substantiate the CPAR. The following areas will be rated on the CPAR as applicable: quality, schedule, cost control, management, and regulatory compliance.

CPARS will be updated, at a minimum, annually, documenting Contractor performance for the contract/order IAW the Guidance for CPARS. Additional information about CPARS can be found in the user manual for CPARS. More frequent updates may be made, as required, to document a significant improvement or decline in the level of Contractor performance.

• The Contractor shall have input into the CPAR prior to it being finalized. For each reporting period, the Government will provide the Contractor a draft CPAR with no ratings, only justifications, to examine for seven calendar days. After MFT review of the Contractor’s response, the CPAR will be coordinated through management up to the Division level. Once approved, the CPAR will be sent to the Contractor via CPARS for the official 60-calendar-day review. The Contractor shall have the option to concur or non-concur, with comments, via CPARS.

• The website for the CPARs toolkit is http://www.cpars.csd.disa.mil/cparsmain.htm.

6.6 Combating Trafficking in Persons

• The place of performance for this effort will be the United States and is considered a low risk location for trafficking in persons. The COR will monitor the Contractor’s performance regarding trafficking in persons on an as-needed basis, such that non-compliance with FAR 52.222-50 is brought to the immediate attention of the CO.

• The COR will watch for signs of trafficking in persons as described in the Combating Trafficking in Persons training located at http://ctip.defense.gov/.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 10 -

7 REVISIONS

Revisions are the responsibility of the MFT. The COR reviews the SOW and QASP annually and shall provide any proposed changes to the MFT for review. Revisions to the QASP are the joint responsibility of the MFT and are only accomplished with the approval of and by the CO.

Specific by-name membership in the MFT will be determined prior to its performance of responsibilities, envisioned to begin in Fiscal Year 2018.

8 SAFETY REQUIREMENTS

8.1 Mishap Notification

• In accordance with AFI 91-204, Safety Investigations and Reports, paragraph 2.4.6.1.1, the Contractor shall report mishaps involving damage or injury to USAF interests.

IAW AFI 91-204, paragraph 1.3.1.1, damage or injury includes damage to DoD property, occupational illness to DoD military or civilian personnel, injury to on- or off-duty DoD military personnel, injury to on-duty DoD civilian personnel, and damage to public or private property or injury or illness to non-DoD personnel caused by USAF operations.

• In accordance with AFI 91-204, paragraph 2.4.6.1.1, the Contractor shall ensure the USAF [Safety Office] and the applicable contract management authority are notified of mishaps. The Contractor shall contact the MFT (the COR, if available, or another MFT member) by telephone within one business day. The MFT will notify the Safety Office and the CO.

8.2 Mishap Investigations

In accordance with AFI 91-204, paragraph 2.4.6.1.2, the Contractor shall cooperate with USAF safety investigations.

8.3 Multi-Functional Team Surveillance

The COR will monitor Contractor reporting and review any mishap notifications. The COR will address safety and/or health issues in the PAR.

- 11 -

QUALITY ASSURANCE

SURVEILLANCE PLAN

Part 2

ASSESSMENT

MANAGEMENT

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 12 -

SERVICES SUMMARY (SS)

PERFORMANCE OBJECTIVES, PERFORMANCE THRESHOLDS,

METHODS OF SURVEILLANCE AND REMEDIES

Performance Objective

SOW

Para.

Performance Threshold Method of Surveillance

Remedy

SS-1 Satisfy Total Non-Mission Capable Supply (TNMCS) rate as part of ICS.

3.9.3.7.6.1

TNMCS rate shall not exceed 5% per month at any Main Operating Base

(MOB).

- 100% review of monthly ICS Data Report

- Periodic surveillance

- CPARS documentation

- Provide CA plan within 5 working days.

SS-2 Maintain issue effectiveness rate at each MOB as part of

ICS.

3.9.3.7.6.2

Issue Effectiveness rate for repairable(s) shall not fall below 85%, and Issue Effectiveness rates for consumable(s) shall not fall below 80% per month at any MOB. Fill Non- Mission Impaired Capable Awaiting Parts (MICAP) Backorders within 72 Hours 90% of the time. Fill MICAPs within 24 Hours.

- 100% review of monthly ICS Data report

- Periodic surveillance

- Customer feedback

- CPARS documentation

- Provide CA plan within 5 working days.

SS – 3

Aircrew Training Devices (ATDs) Certified for training

3.5.3.2.3.2 Maintain all ATDs such that

they are fully certified for training.

Recurring Simulator Certifications

(SIMCERT)

- CPARS documentation

- Provide CA plan within 5 working days.

SS – 4

Trainer Availability

3.5.3.2.5.2 Maintain training device

availability consistent with the APT GBTS Systems Specification.

- 100% review of monthly Contractor Device Performance Report

- Customer Complaint

- CPARS documentation

- Provide CA plan within 5 working days.

SS – 5 Provide Accurate Resolutions to Customer Needs and Agency Requirements

3.5.3.2 Provide at least 90%

acceptable responses to customer needs and agency requirements during each month.

Periodic Surveillance

- CPARS documentation

SS-6 Provide Organizational-Level Field Service Representative (FSR) as part of Sustainment

3.9.3.8 Provide FSR personnel and

subject matter expertise to sufficiently support maintenance operations with technical advisory, supplemental training, and reach-back assistance to support aircraft availability.

- 100% review of monthly Contract Field Service Report

- Periodic surveillance

- CPARS documentation

*The procedures set forth in the Inspection of Services Clause will be used to remedy all deficiencies. The re-performance times will be based upon sensitivity of the issue.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 14 -

1 INTRODUCTION

These surveillance procedures have been developed to implement PBSA. They are designed to provide the COR instructions on how to perform surveillance and document findings on this contract for the APT Program.

1.1 Inspection

The COR may inspect any aspect of Contractor performance at any time during the contract, whether or not it is included in the SS. A format for documenting these inspections is found on page 31 of this plan.

1.2 Defects

Any nonconformance with contract requirements is a “defect.” The term “defective” is used in reference to a service output in the SS that does not meet the outputs associated with the standard.

1.3 Performance Responsibility

This QASP is based on the premise that the Contractor, not the Government, is responsible for the management and quality control actions required to meet the terms of the contract.

1.4 Approach

The following is the approach the team will use to assess the Contractor’s performance against the performance thresholds/metrics identified in the contract.

• The SS will summarize the performance objectives and performance thresholds required by the Government in the Contractor’s performance. These performance objectives will directly relate to mission-essential items. The performance thresholds briefly describe the minimum acceptable levels of services required for each objective and are critical to mission success.

• The COR will perform QA procedures contained herein.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 15 -

2 METHODS FOR SURVEILLANCE PERFORMANCE

2.1 Customer Complaint/Feedback

When the customer (using activity/facility) discovers unacceptable performance or a defect in service, he/she will prepare a Customer Complaint Record (CCR), identifying the discrepancy. The CCR template is included on page 27 of this document. The customer will keep one copy and forward a copy to the COR either manually or electronically. The COR will review the item and determine if the complaint is contractually valid. If valid, it is forwarded to the Contractor for CA. If a customer notifies the COR either verbally or in writing (including email) of a complaint but does not submit a CCR, the COR will generate a CCR to document the complaint. The CCR will then be submitted to the Contractor for

CA.

In addition, the COR or individual customers may from time to time wish to submit positive feedback about the Contractor’s performance under the contract. This positive feedback may be submitted on the Customer Feedback Record (CFR), found on page 26 of this plan.

In the event that the COR wants feedback from every customer/service location each month, those customers with no complaints should submit their feedback on a CFR.

Major complaints are defined as those complaints that do not meet the threshold and/or do not reasonably indicate a deficiency in the service received that would require retraining, recertification, or re-accomplishment of the identified task.

2.2 Periodic Surveillance

When periodic surveillance is used, inspections shall be performed at the frequency identified for each SS item and, otherwise, is on a “100 percent surveillance” or statistically random basis. The frequency of inspection is established (i.e., daily, weekly, monthly, or quarterly) but the specific time of occurrence need not be specified. Random and scheduled inspection will be recorded in a COR Schedule (see attachment, page 24). When the COR discovers unacceptable performance or a defect in service, he/she will generate a CAR (see attachment, page 28) identifying the discrepancy. The CAR will then be submitted to the Contractor for CA.

2.3 100 Percent Surveillance In this surveillance method, the COR inspects 100 percent of the work, through procedures designated in the SS. When the COR discovers unacceptable performance or a defect in service, he/she will generate a CAR identifying the discrepancy. The CAR will then be submitted to the Contractor for CA.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 16 -

3 INFORM CONTRACTOR’S CONTRACT MANAGER

The COR must always contact the Contractor’s manager and inform him/her of any deficiency that was observed. This may be accomplished in person or electronically, but any verbal communication must be supplemented by submission of a CCR or CAR. The Contractor will annotate on the report, provided by the COR, the CA he/she will take to correct the defect and prevent its recurrence. The Contractor will then return the completed report to the COR.

4 EVALUATE CONTRACTOR CORRECTIVE ACTION

After the Contractor documents the proposed CA, the COR will evaluate the proposed CA and forward the record, annotated with the results of the COR evaluation, to the CO for final dispensation. The CO will determine if further actions need to be taken against the Contractor, such as reductions in fee/price IAW the FAR 52.246-4, Inspection of Services, clause. The CO’s disposition of the CA and proposed CA will be annotated on the record, and copies will be returned to the COR and the Contractor.

5 REMEDIES FOR UNACCEPTABLE PERFORMANCE

In accordance with the contract Inspection of Services clause, if any of the services do not conform to contract requirements, the Government may require the Contractor to perform the services again in conformity with contract requirements at no increase in contract amount. When the defects in services cannot be corrected by re-performance, the CO may undertake the following actions:

• Remedial Action. Require the Contractor to take necessary action to ensure that future performance conforms to contract requirements.

• Payment Reduction. Reduce the contract price to reflect the reduced value of the services performed. This may include a reduction in the number of paid hours to reflect the total non-performance period.

• Penalties for Non-Performance. If the Contractor fails to promptly perform the services again or to take the necessary action to ensure future performance is in conformity with contract requirements, the Government may o Government Service Cost. By contract or otherwise, perform the services and charge to the Contractor any cost incurred by the Government that is directly related to the performance of such service.

o Invoice Adjustment. Decrease the invoice amount in relation to performance.

o Cure Notice. Issue a cure notice.

o Show Cause Notice. Issue a show cause notice.

o Terminate the contract. Terminate the contract.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 17 -

The CO will determine the action to be taken against the Contractor, such as reductions in price IAW the Inspection of Services clause. The CO’s disposition of the deficiency will be annotated, and the COR and Contractor will be notified of action.

6 CONTRACTING OFFICER’S REPRESENTATIVE MONTHLY

INSPECTION REPORT

The COR shall submit an assessment report and attach all CARs, PARs, and validated customer complaints to the CO and FC/FD no later than five working days following the month in which the actual surveillance occurred. The report shall be signed and dated by the COR and provided to the FC/FD for review prior to sending the information to the CO. Validated customer complaints and all documentation will be used to support Contractor ratings in CPARS.

7 WIDE AREA WORKFLOW

7.1 Receipt Certification

The COR will certify receipt of Contractor services via the WAWF website. Certification of services shall be accomplished on the first workday following the completion of the contract payment period to avoid payment of interest penalties.

7.2 Wide Area Workflow Training

Contracting Officer’s Representatives are responsible for obtaining, scheduling, and completing WAWF training from the Base Accounting and Finance Office.

8 REVISION TO THE QUALITY ASSURANCE SURVEILLANCE PLAN

Revisions to the QASP are the joint responsibility of the MFT. This is a living document and can be changed at any time by the CO.

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 18 -

9 PERFORMANCE OBJECTIVES, PERFORMANCE THRESHOLDS,

AND METHODS OF SURVEILLANCE

9.1 SS - 1 Satisfy Total Non-Mission Capable Supply Rate

9.1.1 Statement of Work Paragraph:

3.9.3.7.6.1

9.1.2 Method(s) of Surveillance:

• 100 percent review of monthly ICS Activity Report

• Periodic surveillance

9.1.3 Performance Threshold:

As part of ICS, performance is acceptable when five percent or less of possessed time at each MOB and an Initial Operational Test and Evaluation location per month is reported as TNMCS. Performance is unacceptable when TNMCS is greater than five percent at any MOB location per month.

9.1.4 Inspection Procedure(s):

• 100 percent review of monthly ICS Data Report o If the COR validates that performance was unacceptable, the deficiency will be documented on a CAR and submitted to the Contractor’s representative.

Contractor will be requested to sign and date the CAR and state what CA will be taken to correct the deficiency and preclude reoccurrence.

o When the Contractor documents the CA, the COR will evaluate the proposed CA and, if satisfactory, will forward the record to the CO for final dispensation. The CO will, in turn, send copies of dispositioned CAR to the COR and the Contractor for their records.

o When a dispositioned CAR has been received, one copy of the completed CAR will be retained in the QA file maintained by the COR.

• Periodic surveillance o See Pg. 15, Paragraph 2.2, “Periodic Surveillance.”

o See Pg. 16, Paragraph 4, “Evaluate Contractor Corrective Action.”

o See Pg. 16, Paragraph 5, “Remedies For Unacceptable Performance.”

9.2 SS - 2 Issue Effectiveness Rate

9.2.1 Statement of Work Paragraph:

3.9.3.7.6.2

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 19 -

9.2.2 Method(s) of Surveillance:

• 100 percent review of monthly ICS Data Report

• Periodic surveillance

9.2.3 Performance Threshold:

As part of ICS, performance is acceptable when the Issue Effectiveness rate at each MOB per month is reported as greater than 85 percent for reparable(s) and greater than 80 percent for any consumables. Performance is unacceptable when the Issue Effectiveness rate is less than 85 percent for reparable(s) and less than 80 percent for consumables at any MOB. Non-MICAP backorders should be filled within 72 hours 90 percent of the time, and MICAP backorders should be filled within 24 hours.

9.2.4 Inspection Procedure(s):

• 100 Percent Review of Monthly ICS Data Report o See Pg. 18, Paragraph 9.1.4, “100 percent review of monthly ICS Data Report.”

o See Pg. 15, Paragraph 2.2, “Periodic Surveillance.”

o See Pg. 16, Paragraph 4, “Evaluate Contractor Corrective Action.”

o See Pg. 16, Paragraph 5, “Remedies For Unacceptable Performance.”

• Periodic Surveillance o See Pg. 15, Paragraph 2.2, “Periodic Surveillance.”

o See Pg. 16, Paragraph 4, “Evaluate Contractor Corrective Action.”

o See Pg. 16, Paragraph 5, “Remedies For Unacceptable Performance.”

9.3 SS – 3 Aircrew Training Devices Certified for Training

9.3.1 Statement of Work Paragraph

3.5.3.2.3.2

9.3.2 Method of Surveillance:

Recurring SIMCERTs

9.3.3 Frequency:

• Annual certification of each GBTS ATD

• Periodic surveillance

9.3.4 Performance Threshold:

The Contractor shall maintain all ATDs, such that they are fully certified for training.

9.3.5 Inspection Procedure(s):

• SIMCERT support

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 20 -

The Contractor shall support the Air Force-conducted SIMCERT of the ATDs IAW AFI 36-2251, Chapter 6.

o See Pg. 15, Paragraph 2.2, “Periodic Surveillance.”

o See Pg. 16, Paragraph 4, “Evaluate Contractor Corrective Action.”

o See Pg. 16, Paragraph 5, “Remedies For Unacceptable Performance.”

• Aircrew Training Device Performance o The Contractor shall document the performance of the ATDs in Qualification Test

Guides (QTG). The Contractor shall make required QTG flight performance data available prior to each visit to allow review of training device performance.

o The Contractor shall document all identified degradations/deficiencies IAW SOW section 3.5.3.2.3.2, “SIMCERT.”

o At his/her discretion, the COR may periodically survey SIMCERT activities.

o If the COR validates that performance was unacceptable, the deficiency will be documented on a CAR and submitted to the Contractor’s representative. The Contractor will be requested to sign and date the CAR and state what CA will be taken to correct the deficiency and preclude reoccurrence.

o When the Contractor documents the CA, the COR will evaluate the proposed CA and, if satisfactory, will forward the record to the CO for final disposition. The CO will, in turn, send copies of dispositioned CAR to the COR and the Contractor for their records.

o When a dispositioned CAR has been received, one copy of the completed CAR will be retained in the QA file maintained by the COR.

o If correction of the deficiency requires re-performance of the service, the COR shall perform re-inspections to determine if the Contractor adequately re-performed and service is acceptable.

9.4 SS – 4 Trainer Availability

9.4.1 Statement of Work Paragraph

3.5.3.2.5.2

9.4.2 Method(s) of Surveillance:

• Periodic surveillance

• Customer complaint

9.4.3 Frequency:

• Daily

• As required

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 21 -

9.4.4 Performance Threshold:

Maintain training device availability consistent with the APT GBTS System Specification, section 3.5.10, “Reliability and Maintainability.” Operational Availability (AO) shall be at least 95 percent for the ATDs. AO shall be at least 98 percent for all other GBTS devices, including, but not limited to, the aircrew ground egress trainer, ejection seat trainer, and part-task trainers.

9.4.5 Inspection Procedure(s):

• 100 percent review of Contractor Device Performance Report o If the COR validates that performance was unacceptable, the deficiency will be documented on a CAR and submitted to the Contractor’s representative. The Contractor will be requested to sign and date the CAR and state what CA will be taken to correct the deficiency and preclude reoccurrence.

o When the Contractor documents the CA, the COR will evaluate the proposed CA and, if satisfactory, will forward the record to the CO for final dispensation. The CO will, in turn, send copies of the dispositioned CAR to the COR and the Contractor for their records.

o When a dispositioned CAR has been received, one copy of the completed CAR will be retained in the QA file maintained by the COR.

• Customer complaint o See Pg. 15, Paragraph 2.1, “Customer Complaint/Feedback.”

o See Pg. 15, Paragraph 2.2, “Periodic Surveillance.”

o See Pg. 16, Paragraph 4, “Evaluate Contractor Corrective Action.”

o See Pg. 16, Paragraph 5, “Remedies for Unacceptable Performance.”

9.5 SS – 5 Provide Accurate Resolutions to Customer Needs and Agency Requirements

9.5.1 Statement of Work Paragraph

3.5.3.2

9.5.2 Method of Surveillance:

Periodic surveillance

9.5.3 Performance Threshold:

Provide at least 90 percent acceptable responses to customer needs and agency requirements during each month.

9.5.4 Inspection Procedure(s):

Periodic surveillance

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 22 -o See Pg. 15, Paragraph 2.2, “Periodic Surveillance.”

o See Pg. 16, Paragraph 4, “Evaluate Contractor Corrective Action.”

o See Pg. 16, Paragraph 5, “Remedies For Unacceptable Performance.”

9.6 SS – 6 Provide Organizational-Level Field Service Representative as part of Sustainment

9.6.1 Statement of Work Paragraph

3.9.3.8

9.6.2 Method of Surveillance:

• 100 percent review of monthly organizational-level (O-Level) Contract Field Service

Report

• Periodic surveillance

9.6.3 Performance Threshold:

The Contractor shall provide O-Level FSR personnel and subject matter expertise to sufficiently support maintenance operations with technical advisory, supplemental training, and reach-back assistance to support aircraft availability.

9.6.4 Inspection Procedures:

• 100 percent review of monthly O-Level Contract Field Service Report o If the COR validates that performance was unacceptable, the deficiency will be documented on a CAR and submitted to the Contractor’s representative. The Contractor will be requested to sign and date the CAR and state what CA will be taken to correct the deficiency and preclude reoccurrence.

o When the Contractor documents the CA, the COR will evaluate the proposed CA and, if satisfactory, will forward the record to the CO for final dispensation. The CO will, in turn, send copies of dispositioned CAR to the COR and the Contractor for their records.

o When a dispositioned CAR has been received, one copy of the completed CAR will be retained in the QA file maintained by the COR.

• Periodic surveillance o See Pg. 15, Paragraph 2.2, “Periodic Surveillance.”

o See Pg. 16, Paragraph 4, “Evaluate Contractor Corrective Action.”

o See Pg. 16, Paragraph 5, “Remedies For Unacceptable Performance.”

- 23 -

MULTI-FUNCTIONAL TEAM MEMBERSHIP

Proposal Number FA8617-17-R-6219 Type of Service: _________________________________

FC/FD: ___________________ Date: __________________

Chief of Contracting**: __________________ Date: __________________

Contracting Officer: __________________ Date: __________________

Primary COR: __________________ Date: __________________

COR Supervisor: __________________ Date: __________________

Alternate COR: __________________ Date: __________________

QAPC: ___________________ Date: __________________

Program Manager**: ___________________ Date: __________________

Engineer**: ___________________ Date: __________________

Subject Matter Expert**: ____________________ Date: __________________

MFT Contractor*: ____________________ Date: __________________

Designated Official**: ____________________ Date: __________________

Accepted By: ____________________ Date: __________________ Contracting Officer

* Optional - Contractor (signs after award) will have limited participation on the MFT so as to not be involved in any inherently governmental responsibilities.

** As applicable

- 24 -xxxxxx-xx-x-xxxx

CONTRACTING

OFFICER

REPRESENTATIVE

SCHEDULE

WEEK

OF

1-7 Apr

MONTH

/YEAR

Apr 10

Inspection Item →

DAY ↓

Maintain Access to Automated Information

ETC. ---

01 Apr 10 0900

02 Apr 10

03 Apr 10

04 Apr 10

05 Apr 10 0800

06 Apr 10

07 Apr 10

WEEK

TOTAL 2

MONTH

TOTAL 2

Computer-Generated Form

. FOR OFFICIAL USE

ONLY (When filled in)

EXAMPLE OF MONTHLY CONTRACTING OFFICER REPRESENTATIVE

SCHEDULE FILLED OUT

(COMPLETED THE PRIOR MONTH FOR ALL RANDOM INSPECTIONS)

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 25 -

COR SURVEILLANCE CHECKLIST

A = Acceptable U=Unacceptable N/A= Not applicable

METHOD OF

ASSESSMENT-

CONTRACTOR'S

NAME:

CONTRACT

NUMBER:

LOCATION:

Building # WPAFB

A U N/A DATE TIME

QAP

SIGNATURE

SS-1 -

Paragraph

SS-2

SS-3

SS-4

DATE:

Functional Director/Commander Date

EXAMPLE OF COR SURVEILLANCE CHECKLIST

- 26 -

CUSTOMER FEEDBACK RECORD

DATE/TIME OF EVENT

DETAILS OF EVENT

ORGANIZATION

BUILDING NUMBER

INDIVIDUAL

PHONE

NUMBER

NATURE OF COMMENT

CONTRACT REFERENCE

VALIDATION

DATE/TIME CONTRACTOR INFORMED OF OBSERVATION

OTHER COMMENTS OR NOTES

RECEIVED BY (NAME & DATE)

EXAMPLE OF CUSTOMER FEEDBACK RECORD

Quality Assurance Surveillance Plan for the Advanced Pilot Training Program

- 27 -

Customer Complaint Record Template, Feb 05 (SAF/AQCP) One copy each for Complainant, COR, Contractor, and Contracting Officer

CUSTOMER COMPLAINT RECORD

DATE/TIME OF

COMPLAINT

SOURCE OF COMPLAINT

ORGANIZATION

BUILDING NUMBER

INDIVIDUAL

PHONE

NATURE OF COMPLAINT

CONTRACT REFERENCE

VALIDATION

DATE/TIME CONTRACTOR INFORMED OF COMPLAINT

ACTION TAKEN BY CONTRACTOR

RECEIVED/VALIDAT…