Atch_3_-_QASP_(25Oct2012).pdf

36TH RESCUE FLIGHT (AETC) FLIGHT OPERATING INSTRUCTION 21-5

Fairchild Air Force Base WA 99011-9662 25 Oct 2012

Maintenance

Quality Assurance Surveillance Plan (QASP)

COMPLIANCE WITH THIS PUBLICATION IS MANDATORY

OPR: 36

th

RQF/MAQI (MSgt Bergman) Certified by: 36 th

RQF/CC (Maj Marshall)

Supersedes N/A Pages: 47

Distribution: X

(QUALITY ASSURANCE SURVEILLANCE PLAN)

INTRODUCTION

The Quality Assurance Surveillance Plan (QASP) is developed IAW AFI 21-101, Chapter 18, and AETC Sup 1 for Performance-Based Activities, and the Maintenance Contract Performance

Work Statement (PWS). It is designed to provide an effective and systematic surveillance method for fairly evaluating and reporting the Service Provider’s (SP) performance during the contract period.

This Flight Operating Instruction (FOI) uses a combination of surveillance methods to adequately ensure the government of the SP’s acceptable performance.

SUMMARY OF CHANGES

This document is substantially revised and must be completely reviewed.

Revisions to this surveillance FOI are the responsibility of the Chief COR (CCOR).

Revisions to the QASP will be coordinated through the Functional Commander (FC), Contracting Officer (CO) and AETC/A4PM.

Administration of this contract, FA3002-12-R-0022 is the responsibility of AETC CONS/LGCU, Randolph AFB TX.

2 FOI 21-5 25 Oct 2012

Table of Contents

SECTION 1 --- PERFORMANCE ASSESSMENT PLANNING & PREPARATION

1.1. Purpose

1.2. Objective

1.3. Results

1.4. Multi-Functional Team Roles & Responsibilities

1.5. COR’s Role

1.6. Surveillance Coverage

1.7. Surveillance Methods

1.8. Acceptable Performance

1.9. Cost Verification Procedures

1.10. SP Proposal Review Procedures

1.11. Management of Government Furnished Property

1.12. Contract Transition Administration Procedures

1.13. SP Developed Instructions and Procedures

SECTION 2 --- PERFORMANCE ASSESSMENT

2.1. Contract Surveillance Requirements

2.2. Monthly Surveillance Schedule Procedures

2.3. Combating Trafficking in Persons

SECTION 3 --- PERFORMANCE ASSESSMENT RESULTS ANALYSIS

3.1 Non-Conformance Procedures

3.2 Financial Management

3.3 Procedures for verifying statistical information

SECTION 4 --- PERFORMANCE ASSESSMENT REPORTING

4.1 Technical Inspections/Observation Work Areas

4.2 COR End-of-Month Surveillance Summary Reports

SECTION 5 --- PERFORMANCE ASSESSMENT FOLLOW-UP

5.1 CARs for Continual Contractual Non-Conformity

5.2 Letter of Concern (LOC)

5.3 Corporate Discussions

5.4 Financial Withholding

5.5 Cure Notices

5.6 Show Cause Notices

SECTION 6 --- PERFORMANCE ASSESSMENT REPORT CLOSURE

3 FOI 21-5 25 Oct 2012

6.1 General

6.2 Closure of Reports

6.3 Closing out CARs and LOCs

6.4 MFT Evaluation Board (EB)

6.5 CPARS

Attachment 1: Glossary of References and Supporting Information

Attachment 2: Overtime Verification Form

Attachment 3: Travel Verification Form

Attachment 4: Equipment Part Request Form

Attachment 5: Evaluation Report Card

Attachment 6: COR Monthly Surveillance Schedule Template

Attachment 7: COR End-of-Month Surveillance Summary Report Template

Attachment 8: AETC IMT 801, Corrective Action Request, Form

Attachment 9: Initial Performance Review*

Attachment 10: Technical Operations Performance Thresholds

4 FOI 21-5 25 Oct 2012

SECTION 1 --- PERFORMANCE ASSESSMENT PLANNING & PREPARATION

1.1. Purpose

The overall purpose of this QASP is to identify the management approach, methods and tools the multi-functional team (MFT) will routinely use to validate the objectives and goals identified in the PWS. Furthermore, this QASP will identify the strategies, methods and tools the multi-functional team will use to assess the contractor’s performance against the performance thresholds, measurements, metrics, and incentives identified in the PWS.

Specifically this QASP establishes procedures on how this assessment/inspection process will be conducted. It provides the detailed process for a comprehensive oversight process:

What will be monitored

How will monitoring take place

Who will conduct the monitoring

How will monitoring efforts and results be documented

1.2. Objective

This QASP is based on the premise that the SP is responsible for contract management and quality control, not the Government. The Government’s objective in having the service is to provide aircraft maintenance services for the 36 th

RQF at Fairchild AFB, WA.

1.2.1. According to the PWS paragraph 3.1.28 and the Inspection of Services Clause in FAR 52-

246-4, the SP is responsible for implementing and delivering performance that meets contract standards using the government accepted Quality Control Plan. The QASP provides the structure for the government’s surveillance of the SP’s performance to assure that it meets contract standards. It is the government’s responsibility to be objective, fair and consistent in evaluating SP performance in accordance with contractual obligations.

1.2.2. The QASP is not part of the contract nor is it intended to duplicate the SP’s quality control plan. This QASP is a living document and flexibility is required to allow for an increase or decrease in the level of surveillance necessary based on the SP’s performance.

1.2.3. The government will provide a copy of the QASP to the SP to facilitate open communication. However, the QASP should not be considered the end state of the SP quality plan.

1.2.4. The QASP should ensure early identification of performance issues and monitor the process of the SP implemented resolution to minimize impact on mission performance.

1.2.5. ASP Objectives: “to be determined”.

1.3. Results

As part of its acquisition strategy, the Acquisition Strategy Panel (ASP) in collaboration with the

MFT intends to achieve the following results/goals over the full term of this acquisition (1 Oct

2013 through 30 September 2018):

5 FOI 21-5 25 Oct 2012

- $$ cost savings over previous contract

- Specific improved efficiencies, improved customer service etc.

1.4. Multi-Functional Team Roles & Responsibilities

The purpose of the multi-functional team is to create an environment that shapes and effectively executes acquisitions within their purview. The emphasis is on teamwork, trust, common sense, and agility. These stakeholders are responsible for managing the contractor’s conformance with contractual obligations throughout the life of the requirement, instituted under the authority of the senior leadership. The following members will participate in the performance management of this specific contract. Their roles and responsibilities are described as follows:

1.4.1. The Contracting Officer (CO)

Has overall responsibility for overseeing the SP’s performance. The CO, through an assigned

COR, monitors SP performance in the areas of contract compliance, contract administration, cost control, reviewing the COR assessment of the SP’s performance; and resolving all differences between the COR version and the SP’s version of events. The CO is the only person with the authority to direct the SP in the performance of their duties under the contract and to make interpretations of and changes to the contract. The CO and Program Manager (PM) share joint responsibility in facilitating the multi-functional team meetings.

1.4.2. The AETC Program Manager (PM)

Has the overall responsibility for initiating, managing and tracking the various programs that support the contract. The PM:

Is the primary liaison between the CO and the COR

Is responsible for implementing the quality plan/system bid by the contractor

Is responsible for assisting in the preparation of contract requirements and sustainment documents

Is the customers’ single Government PM point of contact

Monitors the SP’s overall performance

Initiates contract modifications

Manages current budget and out-year contract funding requirements

Performs all PWS management actions

Is the focal point for resolutions of all contract technical and functional issues

1.4.3. The Functional Commander/Functional Director (FC/FD)

Has overall responsibility for identifying mission essential services and developing the necessary documents in accordance with DoDI 1100.22 and DFARS 237.76. The FC/FD:

Is the Government’s functional authority for the contracted function (The FC/FD retains all responsibility for the success or failure of the contracted function, the same as if the contracted function was an organic activity)

Is responsible for Continuation of Essential DoD SP Services During Crisis (DFARS

252.237-7023(a)(2) and subpart 237.7602(a))

Is responsible for assigning primary and alternate Contracting Officer Representatives

(COR) who will be available full time or as warranted by the procurement cycle

6 FOI 21-5 25 Oct 2012

Is responsible for reviewing SP performance documentation on a regular basis to ensure performance is compatible with contract and mission objectives while supporting a higher level of SP performance

Is responsible for fostering synergistic partnerships

1.4.4. The COR

The COR is responsible for monitoring the SP’s performance by assessing, recording and reporting compliance with the terms and conditions of the contract. Each COR will be appointed in writing by the CO and complete COR training prior to performing SP contractual compliance surveillance tasks unsupervised. Each COR will maintain surveillance documentation, and notify the Functional Commander/Functional Director (FC/FD), PM and CO, of any significant performance deficiencies, and recommend improvements to the QASP and PWS throughout the life of the contract.

1.4.5. The SP

The SP shall:

Comply fully with the terms and conditions of the contract

Participate as a member of the multi-functional team in the post-award management phase

Maintain and implement a government approved Quality Control (QC) Plan

Ensure non-conforming contract services are identified, root causes are determined, corrective actions are accomplished and follow up actions are taken to ensure no future failures in contractual compliance obligations will occur

SP shall only tender services to the Government that conform to contract requirements.

1.5. COR’s Role

The COR’s role is to observe, then document the overall performance of the SP without duplicating or augmenting the SP’s QC function. Ultimately, CORs protect the government’s interest by being the eyes and ears of the FC and CO. Detailed responsibilities are outlined in

AFI 21-101 AETC Sup 1, Chapter 18, para 18.7. It is vitally important CORs identify areas that do not meet contract performance standards/requirements. CORs must remain informed of all maintenance operations and maintain a relationship based on mutual respect. Some COR rules to follow include:

Be honest and fair

Be consistent with surveillance activities

Communicate accurate information

Do not make your surveillance activities personal

Do not get emotional, never argue with the contractor employees

Never train or advise the contractor

Keep senior leaders informed

Do not become the SP’s QC

Do not pursue/facilitate unprofessional relationships

Document all discovered defects/discrepancies

7 FOI 21-5 25 Oct 2012

1.5.1. Chief Contracting Officer Representative’s (CCOR) Role

The CCOR ensures performance is surveilled IAW criteria outlined in the PWS and this QASP.

CCOR will report noncompliance or abnormalities to FC and CO. In CCOR absence, COR will serve as CCOR. Detailed responsibilities are outlined in AFI 21-101 AETC Supp 1 Chapter 18 paragraph 18.6.

1.5.2. COR Training

Training will be conducted IAW the COR training plan and consist of the mandatory requirements identified in IAW MP5301.602-2(d), MP5346.103, AFI 21-101 & AETC

Supplement and requirements locally determined by the FC, Chief COR, Superintendent, Quality

Assurance Program Coordinator (QAPC), CO. Individual training records will be maintained

IAW MP5301.602-2(d), AFI 21-101 & AETC Supplement & AFI 36-2201, Vol. 3.

1.5.2.1. Initial Training:

1.5.2.1.1. As required by Mandatory Procedure Air Force Federal Acquisition Regulation

Supplement (AFFARS) (MP) 5301.602-2(d) and AFI 21-101, AETC Sup, CORs must complete the following Defense Acquisition University (DAU), AETC and local training prior to performing surveillance duties:

COR 222, Contracting Officer’s Representative course (DAU) CLM 003, Ethics Training for Acquisition Technology and Logistics course (DAU) 393AET0066-002, AETC COR Course (AETC, within 90 days of assignment) 393AET0066-003, AETC CCOR Course (AETC, within 90 days of assignment) Quality Assurance Program Coordinator (QAPC) led training (accomplished by

393AET0066-002)

CO led training

The FC /Chief COR will ensure required TDY funding is projected in the unit budget to ensure all COR related training requirements are met.

1.5.2.1.2. To ensure adequate coverage, CORs are also trained in areas of responsibility, and duties outside their normal AFSC by cross utilization training (CUT). AF & AETC directives and technical orders will be used to complete on the job training (OJT) in areas where the individual COR lacks extensive knowledge. Alternate CORs will maintain currency on those evaluation guides they are assigned as alternate inspector.

1.5.2.1.3. Newly assigned COR’s will study the required directives and perform the observation guide under the supervision of a qualified trainer before being certified in a specific area. The trainer will conduct OJT until the new COR demonstrates satisfactory knowledge of each work area and surveillance techniques. Training will be documented on Individual Training Plan (ITP) in the automated Training Business Area (TBA).

1.5.2.1.4. In addition to previously mentioned COR training, the Chief COR/COR

Superintendent will attend the AETC Chief COR course 393AET0066-001 immediately following completion of course 393AET0066-002, but not later than 90 days of assignment.

8 FOI 21-5 25 Oct 2012

1.5.2.1.5. The Functional Commander will successfully complete Functional Commander’s

Executive Training Session within 90 days of assignment.

1.5.3.2. Refresher Training:

1.5.3.2.1. All CORs will accomplish HQ AETC/A4MMR developed CBT Annual Refresher

Training NLT 120 days after posting (October each year).

1.5.3.2.2. All CORs will annually review and understand the SAF/GCA Air Force Ethics

Training Slides.

1.5.3.2.3. All CORs will annually re-accomplish DAU course CLM 003, Ethics Training for

Acquisition Technology & Logistics.

1.5.3.2.4. CORs will re-accomplish course 393AET066-002, Block I, in-residence every three years. This satisfies the annual CBT refresher training requirement for that year. The Chief

COR/Superintendent will re-accomplish course 393AET0066-001 in-residence course every three years which will also satisfy the annual CBT refresher training requirement for that year.

1.5.3.2.5. Every three years, all CORs will accomplish DAU course COR 222, Contracting

Officer Representative (COR) with a mission focus.

1.5.3.3. COR Training Records:

1.5.3.3.1. As a minimum, all CORs (military and civilian) must maintain training records regardless of grade or skill level identifying specific responsibilities. All CORs must maintain an ITP in the automated Training Business Area (TBA)

1.5.3.3.2. COR Tracking (CORT) Tool. The Chief COR and CO will ensure that all COR training will be entered into the CORT Tool database.

1.5.3.4. Initial Evaluations:

1.5.3.4.1. The Chief COR will accomplish an initial evaluation on each COR to determine past qualifications, experience, and ability to accomplish technical inspections and contract surveillance functions within 15 days of assignment. Any deviations between position requirements and new CORs verified skill-set as a result of the initial evaluation will be documented and in the new CORs ITP. Initial evaluations of all CORs will be documented in

TBA.

1.5.3.4.2. The Chief COR performs initial and annual over-the-shoulder (OTS) evaluations of

COR personnel to ensure proficiency in surveillance techniques, ensuring COR personnel are knowledgeable of the tasks they surveil. If the Chief COR performs surveillance duties, the FC will perform an initial and annual OTS on the Chief COR. All initial and annual OTS evaluations will be documented in TBA.

1.6. Surveillance Coverage

Duty hours will be flexible to ensure complete coverage of SP’s performance on both shifts.

9 FOI 21-5 25 Oct 2012

1.6.1. SP hours are normally from 0700-1500 and 1500-2330 local. CORs will perform surveillances on the second shift on a periodic basis. Work hours may be adjusted appropriately to accurately assess performance on second shift.

1.6.2. As a minimum, CORs will complete 10% of its surveillance of the SP’s second shift. This will be a shared responsibility between the CCOR and COR.

1.6.3. Periodically the weekend crew will be surveilled not less than once a quarter.

1.6.4. Periodically the Cusick maintenance team at Cusick will also be surveilled not less than once a quarter.

1.7. Surveillance Methods

This contract is a Firm Fixed Price “C” type contract. Services are negotiated and the SP invoices are paid in monthly installments with the exception of the few cost reimbursable CLINS that are incidental to the performance of this contract. Services are inspected by the government to ensure services rendered are in compliance with the PWS and the established thresholds for this effort. The following are methods used performing surveillance:

1.7.1. Performance Metrics

Aircraft maintenance performance metrics are reported to higher headquarters and are summarized in the Technical Operations Performance Thresholds listed in the Services Summary

(See Attachment 10 and PWS Section 3-2). COR personnel track these metrics and validate the accuracy.

1.7.2. Periodic Inspection

These services are inspected on a periodic and reoccurring basis. Periodic surveillance may be daily, weekly, monthly, quarterly, etc.(See below for list of planned periodic inspections)

Technical Inspections

Preflight

Thruflight

BPO

Support Equipment / AGE

FOD (Aircraft)

FOD (Hangar/Flightline/Offices)

CTK

Sheetmetal

Liquid Servicing (fuel service)

Ground Movement

Flight Control/Rig

Landing Gear Maintenance

Throttle Rig

Aircraft Wash/Corrosion

10 FOI 21-5 25 Oct 2012

E&E Maintenance

Avionics Maintenance

Propulsion Maintenance

Fuel System Maintenance

Document File Inspection (Aircraft Jacket File)

Document File Inspection (Engine Records)

Weight & Balance (Aircraft)

Tug Inspection

Observation Area Inspections

Aircraft Flight Line

-21 Equipment

Quality Control

Data Management

FCF

Cusick Operations

Debrief

Aircraft Scheduled Maintenance Facility

Wash Rack

Engine

Training Management

Haz Com/Material

Technical Orders

CTK

Maintenance Supply Liaison

Documentation

Plans and Scheduling

Maintenance Operation Center

Training (task qual and cert training)

1.7.3. Customer Complaint

The COR is the primary point of contact for collection of all customer complaints. All complaints and any resulting resolution of such complaints must be documented on the form.

Customer complaint forms become a permanent part of the COR surveillance records.

1.7.4. 100% Inspection

These items are inspected on every instance they are performed. The COR will determine acceptability and compliance with the performance threshold by monitoring and tracking each deliverable document. Technical documents and other deliverable documents will be reviewed for technical accuracy and compliance with technical directives. The results of this inspection may be used as the basis for actions (other than payment deductions) towards the SP. In such cases, the Inspection of Services clause becomes the basis for the contracting officer’s actions.

(See below for planned list of 100% inspections)

11 FOI 21-5 25 Oct 2012

TCTO

AETC Special Inspection

Transfer/Acceptance Inspection

Engine Installation or Combining Gearbox (CGB)

Phase Inspection

Post Dock

1.8. Acceptable Performance

This is determined by evaluating contactors ability to meet requirements outlined in Section 3-2

“Service Summary” and performance levels established in their quality plan. In addition, COR office will partner with SP QC to establish an Acceptable Quality Level (AQL) for all technical and observation area inspections. It is key for COR to determine if the SP has an effective QC

Program when determining acceptable performance.

1.8.1. Defining attributes of an effective quality program:

1) Clearly defined independent plan that promotes a proactive approach throughout the workforce.

2) Inspection process that effectively measures how well the maintenance organization meets/exceeds standards.

3) Fosters the production generation of safe reliable equipment that meets/exceeds mission requirements.

4) Ensures all contractual requirements and obligations are met including customer satisfaction at the lowest overall cost while continuing to improve their process.

5) Implemented 5 quality elements into a developed quality program.

I IDENTIFY “detects errors”

II ANALYZING “identifies negative trends”

III CORRECTING “implements methods for improvements and correcting root cause”

IV DETERMINE EFFECTIVENESS “follow up on adequacies of corrections”

V DETECTS POTENTIAL DEFICIENCIES “active self inspection program”

1.8.2. General guides

General Guides will be used to cover critical items contained in AFI 21-101, AETC sup 1 and special areas of the PWS in the accomplishment of Observation Area inspections. These general guides will be used in conjunction with technical and observation surveillances guides.

Discrepancies identified with general guides can be annotated with primary evaluation guide findings or separately.

1.8.3. Evaluation guides

Evaluation guides will be created for all observation areas and reviewed once a year at different monthly increments by the CCOR. The guides will be derived from the PWS to include regulations referenced from publications in appendix 2A and SP procedures and QC Plan. The

CCOR will review procedures ensuring they meet the intent of PWS and affiliated regulations.

12 FOI 21-5 25 Oct 2012

1.8.4. Locally Determined Minor Discrepancy Baseline Standards

After SP develops QC Plan, COR office will partner with QC to determine AQL for established inspections.

1.9. Cost Verification Procedures

COR’s must review SP cost requests to ensure accuracy and relevance. COR will be required to review costs associated with overtime, travel, equipment parts and vehicle fuel.

1.9.1. Overtime

1) SP sends request on Overtime Verification form (attachment 1) and completes the estimate portion of the form.

2) CCOR office reviews form for accuracy, ensures the request looks reasonable and funding is available to cover costs.

3) After review, forward verification form for estimate approval to Contract Specialist (CS) / CO and courtesy copy FC.

4) After approval, send Site Manger form with CO approval.

5) Upon completion of overtime and end of SP pay week, review SP’s completed Overtime

Verification form. Ensure remarks are included detailing differences between amounts requested. COR should occasionally physically verify employees are present during overtime work and review SPs timecards.

6) After review, forward verification form for approval to CS / CO and courtesy copy FC.

7) After approval, send Site Manger completed form with CO approval.

8) Annotate overtime registry for amount-authorized reimbursement via WAWF.

1.9.2. Travel

1) SP sends request on Travel Verification form (attachment 2) and completes the estimate portion of form.

2) COR office reviews form for accuracy, ensures the request looks reasonable and funding is available to cover costs. Pay particular attention to per diem rates authorized using Defense

Travel Management website: http://www.defensetravel.dod.mil .

3) After review, forward verification form for approval to CS / CO and courtesy copy FC.

4) After approval, send Site Manger form with CO approval.

5) Upon completion of travel, review SP’s completed Travel Verification Form. Ensure remarks include a detailed explanation of any differences between amounts requested.

6) After review, forward verification form for final approval to CS / CO and courtesy copy FC.

7) After approval, send Site Manger completed form with CO approval.

8) Annotate travel registry for amount-authorized reimbursement via WAWF.

1.9.3. Equipment Parts

1) SP sends request on Equipment Part Request Form (attachment 3).

2) COR office reviews form for accuracy, ensures the request looks reasonable and funding is available to cover costs. Investigate the part requiring replacement was not damaged by SP neglecting item.

3) COR will add comments to form and send back to Site Manager with approval/disapproval to purchase part.

http://www.defensetravel.dod.mil/

13 FOI 21-5 25 Oct 2012

4) Upon SP purchases part (if approved), request receipts and log cost in equipment part registry for reimbursement via WAWF.

5) COR’s will validate that equipment/parts and or tools requested for purchase by the contractor are incorporated into applicable inventories IAW PWS requirements.

1.9.4. Vehicle Fuel

1) At end of month, SP provides invoice of fuel transactions to COR office.

2) COR office reviews invoice for validity, accuracy, and ensures funding is available to cover costs. Verify fuel used matches expected use by comparing number of gallons used vs. miles drove.

3) Log fuel cost in Fuel Cost Reconciliation registry for reimbursement via WAWF.

1.10. SP Proposal Review Procedures

SPs may request changes to PWS or find a way to save government money by improving a process.

1.10.1. Change Proposals

COR’s will review and give recommendation to the FC and CO on SP proposals. Evaluation of proposal will ensure changes do not contradict other instructions / regulation. In addition, evaluation will ensure the change is compatible with 36 th

Rescue Flight’s mission. Change proposals can be submitted by memorandum or e-mail to the COR personnel. The COR personnel have 30 days to review/research and submit recommendations to the FC and CO for final evaluation and approval. The A4PM office will also be courtesy copied for input in the evaluation process and to provide comments. All comments will be forward with the use of e-mail. The CORs will keep the SP informed of the status concerning the change proposal through e-mail. All correspondence regarding proposals will be filed in COR contract files.

1.10.2. Value Engineering Change Proposals (VECP)

COR personnel will review VECP submitted by the SP IAW FARs 48 and 52.248. COR personnel have 15 working days to assess the proposal and submit recommendations to the FC and CO. If COR personnel cannot meet the 15 working day time limit, the COR must inform the proposal originator of the current status (weekly) and when it is projected to be completed.

1.11. Management of Government Furnished Property

1.11.1. Government furnished property includes but is not limited to Equipment Authorized

Inventory Data (EAID) and non-EAID (common hand tools, furniture, workbenches, etc) and shall be inventoried by SP on an annual basis.

1.11.2. A joint inventory of Government furnished property will be conducted biannually with

COR and SP personnel. Completion of the joint inventory will be documented on a

Memorandum for Record signed by the CCOR and Site Manger (or their designees).

1.11.3. Results of the inventories will be provided to the CO IAW the provisions and requirements of the PWS.

14 FOI 21-5 25 Oct 2012

1.11.4. See FAR Part 45 for information concerning government property.

1.12. Contract Transition Administration Procedures

Upon contract award, COR office will acquire SP’s transition plan and inform SP it will be monitored. Discrepancies will be documented on AETC IMT 447 as appropriate and forwarded to FC and CO for action. During the transition month a monthly summary will be created for both the incumbent and successor SPs and sent to CO, FC, and A4PM.

1.13. SP Developed Instructions and Procedures

CCOR will review changes to applicable directives to assure the SP is obtaining end results intended by the changed directives. After review, CCOR routes directive to the FC for acceptance with copies forwarded to A4PM and CO. Specific attention will be given to contractor’s quality plan.

15 FOI 21-5 25 Oct 2012

SECTION 2 --- PERFORMANCE ASSESSMENT

2.1. Contract Surveillance Requirements

Inspections are broken down into two categories, Technical and Observation. While some evaluation guides have been developed for the technical surveillances, all observations surveillances will have a guide.

2.1.1. Technical Inspections

COR personnel perform technical inspections on all SP maintained aircraft, engines and support equipment. Any maintenance task accomplished IAW technical guidance (TO, work-card, etc.)

qualifies for COR surveillance under the technical inspection concept.

2.1.1.1. COR personnel must perform at least 50% of all inspection items for each technical inspection accomplished. When evaluating only a portion of the required inspection items ensure documentation includes specifically identifying/annotating those items that were evaluated.

2.1.1.2. COR personnel review the aircraft or equipment forms and automated maintenance information system (MIS) for accurate and valid documentation applicable to the inspection being surveilled.

2.1.1.3. COR personnel check for proper and current tech data usage, proper tool usage, and after maintenance Foreign Object (FO) checks of the area in which the task was performed.

Discrepancies identified in these categories are applied to the technical inspection.

2.1.1.4. COR personnel will ensure all inspections are documented in the local COR database and the report is emailed to the site manager. When a discrepancy is noted, an electronic AETC

IMT 447 will be also emailed to site manager with request to digitally initial. Give SP a reasonable suspense to answer 447 not to exceed 7 calendar days.

2.1.1.5. COR personnel perform scheduled technical inspections at the first available opportunity and are responsible to ensure their assigned inspections are completed no later than the last day of the month.

2.1.1.6. Violations of OSHA or AFOSHSTDs that clearly present a potential to damage or injure government resources are documented as part of the inspection being performed or, if appropriate, “As Observed”. Documentation should clearly indicate the potential to damage or injure government resources. CORs do not document violations of OSHA or AFOSHSTDs that do not present the potential to damage or injure government resources; rather they will informally notify the site supervisor and CO.

2.1.1.7. Aircraft document file inspections shall include a review of the status and historical documents to including documents in MIS.

2.1.1.7.1. Discrepancies of a historical nature, including automated documents that can be verified from other sources are to be corrected by the SP.

16 FOI 21-5 25 Oct 2012

2.1.1.7.2. The correct use and clearance of Red X symbols are items of special attention during document file inspections. Each incorrect clearing of a Red X symbol, erasure of symbols, overdue TCIs and overdue inspections caused by improper documentation are considered major discrepancies.

2.1.1.7.3. CORs must ensure unsafe or unfit for operation conditions are represented by Red X entries and that these entries are properly cleared in aircraft or equipment forms and Maintenance

Information Systems IAW PWS.

2.1.2. Observation Inspections

These inspections are accomplished by assessing a work center’s/area’s ability to manage program areas. Observation work areas will be surveilled by performing inspections within each work area utilizing observation evaluation guides and the general observation guide.

2.1.2.1. CORs will provide references for discrepancies entered on AETC Form 447. CORs should avoid areas that are gray. Do not nitpick or be forgiving with the inspection tone. CORs should surveil IAW the specific provisions of the PWS focusing on the SP’s accomplishing quality maintenance in a timely manner.

2.1.2.2. A general evaluation guide will address the following critical items: tool and equipment management, FO prevention, housekeeping procedures, supply procedures, technical order maintenance, forms documentation, MIS documentation, physical security, utilities conservation, fire prevention, environmental protection and facilities management.

2.1.2.3. A rating of conforms or non-conforms will be assigned based on whether or not the training observed meets contract requirements, the standards and intent of the SP’s training plan and the objectives of the training being provided.

2.1.2.4. COR personnel will ensure all inspections are documented in the local COR database and the report is emailed to the site manager. When a discrepancy is noted, an electronic AETC

IMT 447 will be also emailed to site manager with request to digitally initial. Give SP a reasonable suspense to answer 447 not to exceed 7 calendar days.

2.1.3. Unscheduled Inspections

These are specific inspections CORs perform outside or above the inspections listed on the FC reviewed monthly surveillance schedule. Prior to accomplishing unscheduled inspections, the

CCOR will coordinate with the FC and CO. The site manager will be informed as to the nature and quantity of unscheduled inspections.

2.1.3.1. Discrepancies will be loaded in local COR database and documented on digital AETC

IMT 447. Email AETC IMT 447 to site manager for digital signature and answer to finding(s).

2.1.3.2. Findings will be non-rated and will not be used to determine performance acceptability for the SP’s monthly or quarterly rating.

17 FOI 21-5 25 Oct 2012

2.1.4. As Observed Inspections

These inspections are observations that are not directly associated with a scheduled or unscheduled inspection.

2.1.4.1. Discrepancies will be loaded in local COR database and documented on digital AETC

IMT 447. Email AETC IMT 447 to site manager for digital signature and answer to finding(s).

2.1.4.2. Findings will be non-rated and will not be used to determine performance acceptability for the SP’s monthly or quarterly rating.

2.1.5. Customer Complaint

2.1.5.1. Customer complaints will be routed through COR office. The COR personnel will request the complainant document nature of their complaint on a Customer Complaint Record

(SAF/AQCP Form). CORs will then determine the legitimacy. If legitimate, the CCOR will forward Customer Complaint Record (SAF/AQCP Form) to the FC and then CO for determination of required action. After CO review, it will be sent to SP for corrective action.

After completion by the SP, the CCOR will forward a copy to the complainant, SP, CO and file a copy in COR file. It will be the discretion of the CCOR if validation of corrective actions or follow-ups is necessary to determine if root cause has been corrected.

2.1.5.2. Each month the COR will contact government customers involved with this contract to assure there is an understanding of the contract requirement by all appropriate personnel.

2.2. Monthly Surveillance Schedule Procedures

2.2.1. The CCOR will identify at least two technical inspections per month as SP QC follow-up inspections. The monthly technical inspection schedule will have the QC follow-up/concurrent inspections identified with an asterisk or highlighted in bold print. The CCOR may elect to do

QC follow-up/concurrent inspections on observation work areas as well.

2.2.2. The monthly surveillance schedule will be completed and forwarded to the FC not later than 5 duty days prior to the beginning of the period it covers. The FC must review and return the schedule to the COR no later than the last day of the month proceeding the scheduled month.

2.2.2.1. A copy of the FC reviewed monthly surveillance schedule will be provided to the CO before the start of the surveillance period.

2.2.3. The surveillance schedules will be marked “FOR OFFICIAL USE ONLY”. Do not release to anyone other than authorized government personnel. These records are exempt from release under FOIA Exemption High (b) (2).

2.2.4. CCOR shall base assessment or inspection on past performance, mandatory, statutory and regulatory requirements.

2.2.5. Performance assessment planning shall consider operational risk, service complexity and criticality as factors in developing the surveillance schedule from month to month. COR will focus on areas that caused the non-conforms rating, and document all discrepancies on an AETC

18 FOI 21-5 25 Oct 2012

IMT 447. If unscheduled, the inspection will not be used to rate SP performance. COR personnel will re-inspect (but are not limited to) those areas that caused the unacceptable rating and document any deficiencies for trend analysis.

2.2.5.1. If a particular function of the SP’s performance has a continuing record of conforms performance and non-conforms performance would not likely result in loss of life to AF personnel or damage to government property, surveillance frequency of that function may be reduced.

2.2.5.2. If SP performance of a function is less than satisfactory, surveillance of that function should be increased. The CCOR has the authority to increase the number of monthly inspections to adequately evaluate areas of SP performance identified as non-conforms. Chief COR will coordinate FC and CO approval when the surveillance schedule requires adjustment as identified above.

2.2.5.3. Standards identified in the Services Summary (SS) will be utilized during the development of technical inspection requirements.

2.2.6. The COR is responsible for developing a long-range annual schedule to ensure adequate surveillance of monthly and quarterly requirements. The COR will review areas of responsibility to ensure no areas are missed and to prevent contractor predictability of upcoming inspections.

2.2.7. Changes to the electronically signed monthly surveillance schedule will include a brief justification and will be coordinated through the Chief COR for review and forwarded to the FC and CO for approval.

2.2.8. If surveillance is not required in an observation area based on a continuing record of acceptable performance, the Chief COR will obtain approval from the FC and CO and will annotate the monthly summary. NOTE: Surveillance of an Observation Work Area will not be skipped more than one prescribed period (monthly, quarterly, etc.)

2.2.9. INITIAL PERFORMANCE REVIEW (IPR). The initial evaluation of the service provider’s performance shall take place within 30 days after the service provider assumes full performance responsibilities (i.e. after completion of transition/mobilization) to ensure the service provider has successfully started performance, completed transition, is fully operational, and is within the estimated cost, schedule, and performance parameters of the contract.

2.2.9.1. INITIAL PERFORMANCE REVIEW CRITERIA. See attached Initial Performance

Review Report (Attachment 8)

2.3. Combating Trafficking in Persons

DoD has a zero tolerance policy for human trafficking. CORs are the first line of defense in the battle against human trafficking and must complete Combating Trafficking in Persons training annually. As the COR monitors the contractor and its employees, Combating Trafficking in

Persons should rank among the COR’s chief priorities. CORs must be diligent in ensuring that contractors and contractor personnel are not trafficking in persons. During surveillance, the

19 FOI 21-5 25 Oct 2012

COR will monitor the contractor’s performance regarding trafficking in persons for compliance with all that is required IAW FAR clause 52.222-50, Combating Trafficking in Persons. The

COR must inform the Contracting Officer if the contractor, contractor personnel, subcontractor, or subcontractor personnel have failed to comply with the requirements of the clause at FAR

52.222-50.

Note: The COR must report any suspected violations or activities to the Contracting Officer (See

FAR Subpart 22.17 and DFARS Subpart 222.17).

Note: CORs should not personally investigate suspected incidents of Trafficking in Persons but should forward all reported or suspected violations to the Contracting Officer immediately.

Note: The Contractor must take appropriate action should a violation occur. For corrective procedures and remedies for non-compliance, see FAR 52.222-50.

20 FOI 21-5 25 Oct 2012

SECTION 3 --- PERFORMANCE ASSESSMENT RESULTS ANALYSIS

3.1 Non-Conformance Procedures

After the monthly COR report is completed (see section 4.2), any technical inspection area or observation work area not meeting PWS Service Summary section 3-2 standards must have a

Corrective Action Request (CAR) initiated by the CCOR as required by AFI 21-101, para 18.15.

A CAR can also be issued for other areas beyond the Service Summary within the contract where the SP’s obligations were not met. This can include significant negative trends or individual

PWS requirements either of which have the potential to threaten or degrade the mission.

3.1.1. When performance is rated non-conforms for the respective rating period as identified in

AFI 21-101, para 18.15, it will be identified and listed as a negative trend on the COR trend tracking list.

3.1.2. When a CAR is initiated, it is forwarded through the FC to the CO for evaluation. If the

CO determines it is appropriate, the CO will issue the CAR to the SP with return receipt requested. All CARs will be initiated using the Corrective Action Request Form AETC IMT 801

(see Attachment 7).

3.1.3. The SP has 15 calendar days from the date of receipt to return the CAR to the ACO with a response as to cause, corrective action, and actions taken to prevent recurrence and reasonable get-well date. If SP fails to correct discrepancy by projected get-well date, COR will document and forward failure to FC for routing to CO for further action. CO may determine to initiate another CAR or, depending on severity, initiate a LOC.

3.1.4. The CCOR will assist the CO with evaluation of the SP’s proposed corrective action and preventative action plans for acceptability.

3.2 Financial Management

3.2.1. Wide Area Work Flow (WAWF) invoicing

Certification of services is performed by the COR at the end of each period of performance via

WAWF system. At the end of each billing period the SP inputs invoice information in the

WAWF system. The WAWF automatically notifies the appropriate COR via email of pending invoice. The COR accesses WAWF and verifies accuracy of the SP’s data. If data is correct the

COR certifies invoice and payment through the Defense Finance Accounting System. If the

COR encounters errors in the SP’s invoice, the COR will resolve with the CO/PM. If the SP disagrees with the COR the issue is forwarded to the CO for resolution. The COR will maintain a copy of the invoice in their respective contract files.

3.2.2. Cost Reimbursable Expenses

CORs are responsible IAW PWS for Contract line item for travel, costs for travel per-diem fees for conferences, seminars, classes, and the items associated with Para 2.1.2 of the PWS are reimbursable expenses. These items are only reimbursable upon approval from the contracting

21 FOI 21-5 25 Oct 2012 officer before the expense is incurred. All information regarding reimbursable travel expenses will be filed in COR contract files under travel.

3.3 Procedures for verifying statistical information

The COR performs evaluations of statistical information provided by the SP relating to mission performance thresholds by reviewing data input while performing technical and observation area evaluations that require data input. In addition, COR verifies data entered on 7501 is accurate by comparing data entered in IMDS screens 460, 179 and 362. Particular attention is paid to items that are SS requirements.

22 FOI 21-5 25 Oct 2012

SECTION 4 --- PERFORMANCE ASSESSMENT REPORTING

4.1 Technical Inspections/Observation Work Areas

CORs will document all inspections in local COR database. When discrepancies are noted, complete digital AETC IMT 447 and route report through CCOR for forwarding to the SP to allow comments to be included for discrepancies. All findings must have specific PWS and tech data references applied. COR will ask SP’s Site Manager to apply digital signature on AETC

IMT 447 to acknowledge the discrepancy.

4.1.1. Ratings: Each scheduled inspection will be assigned a rating of conforms or non-conforms.

4.1.1.1. Conforms ratings are assigned when no major discrepancies are identified and the total number of minor discrepancies does not exceed the applicable baseline or AQL.

4.1.1.2. Non-conforms ratings are assigned when a major or Red X discrepancy is detected; a step is omitted or improperly completed or serious enough to adversely affect the performance of the equipment involved; the performance threshold has not been met or minor discrepancies exceed the established baselines.

4.1.2. Major and minor discrepancies are defined in AFI 21-101 and AFI 21-101/AETC

Supplement 1, Chapter 18. Baselines for minor discrepancies are published in the QASP.

However, the baseline for all observation work area evaluations is 6 minors.

4.1.2.1. Obvious defects, which could have been readily detected by a technician or supervisor, but is not a specific work card item or TO step will be documented for trend analysis, but will not count against the AQL.

4.1.3. Non-rated ratings are assigned to unscheduled or as observed reports and will not be counted towards the performance threshold standards.

4.1.4 CCOR verifies monthly and quarterly performance thresholds were met by comparing SP submitted data against COR and wing analysis data.

4.2 COR End-of-Month Surveillance Summary Reports

As required by AFI 21-101, para 18.14, the CCOR will forward the end-of-month surveillance summary to the FC, CO, AETC A4PM and the AETC COR Functional Manager not later than the 15 th workday after each monthly reporting period is complete.

4.2.1 The Surveillance report will include the items as identified in the end-of-month surveillance report template found in Attachment 5.

23 FOI 21-5 25 Oct 2012

SECTION 5 --- PERFORMANCE ASSESSMENT FOLLOW-UP

5.1 CARs for Continual Contractual Non-Conformity

5.1.1. If the corrective action cited by the SP in their response to the CAR fails to correct the non-conformance, the CCOR shall issue another CAR for any subsequent surveillance rating periods in the same non-conforming area.

5.1.2. During periods when a CAR remains open because the SP self-identified “get-well” date has not been reached, if regularly scheduled COR surveillance is still required, CORs will still accomplish this surveillance and identify any defects. However, CORs must annotate on the

AETC IMT 447 that the item still remains open and document the progress of the SP’s progress in correcting the non-conformity.

5.1.3. Defects detected by CORs during the get-well period will still be charged to the SP and included in the monthly rating reflected on the.

5.1.4. CORs will implement follow-up inspections to confirm the effectiveness of SP implemented corrections to verify if solutions truly address root cause(s) and eliminate the contractual non-conformity. Once the CCOR is satisfied the SP has corrected the non-conformity and adequately addressed the root cause(s), they will pursue closeout of the CAR as specified in Section 6 of this QASP.

5.2 Letter of Concern (LOC)

LOC memorandums are issued to notify the SP corporate office that the government is not satisfied with the local SP’s efforts to date to correct previously identified contractual non-conformities via a CAR.

5.2.1. LOCs are issued typically after two or more previously issued CARs identifying the same contractual non-conformance has not been remedied by local SP corrective measures.

5.2.2. The LOC will be a memorandum issued by the FC and the CO or Procurement Contracting

Officer (PCO) for the purpose communicating in writing to the SP corporate offices that local SP efforts have been unsuccessful in remedying prolonged contractual non-conformities identified in previously issued CARs. The LOC will inform the SP corporate offices that more aggressive corrective actions must be undertaken to remedy the contractual non-conformity.

5.2.3. As with CARs, CORs will continue to implement follow-up inspections to track the progress/status of SP initiated corrective actions.

5.3 Corporate Discussions

Corporate discussions will be used when the FC and CO are not satisfied with the contractor’s local management in remedying issues previously addressed via a CAR or LOC. The end goal of corporate discussions is to resolve significant contractual nonconformance issues before they become disastrous and negatively affect the mission.

24 FOI 21-5 25 Oct 2012

5.3.1. It is highly encouraged that before the FC and CO initiate a discussion involving corporate headquarters that they contact the AETC PCO and the applicable AETC/A4PM functional for the contract.

5.3.2. It is highly encouraged that the PCO and AETC A4PM facilitate corporate discussions.

5.4 Financial Withholding

Currently, Performance-Based Service Acquisition (PBSA) concepts provide SPs the opportunity to re-perform services not meeting contract standards and/or allow the SP the opportunity to take necessary action to ensure that future performance conforms to contract requirements. It should also be noted that financial withholding is to be used only as a last resort after attempts at other corrective actions have been used. Initiation of CARs, LOCs and corporate discussions serves this purpose and satisfies the intent of PBSA.

5.4.1. Financial withholding should only be pursued by the CO AFTER the following three circumstances have occurred:

5.4.1.1. When a CAR, LOC and corporate discussions have not resolved the contractual nonconformance within a reasonable period of time.

5.4.1.2. When continued nonconformity has the realistic potential to jeopardize mission accomplishment.

5.4.1.3. When the PCO and AETC/A4PM concur.

5.4.2. The following FAR references provide the foundation for COs to withhold payment for non-conforming services:

5.4.2. 1. FAR 46.102(b) - Agencies will ensure that supplies or services tendered by SPs meet contract requirements

5.4.2. 2. FAR 46.105(a)(2) – The SP is responsible to tender for acceptance only those services that conform to contract requirements - Far 52.212-4

5.4.2. 3. FAR 52.212-4(i) - Payment shall be made for items accepted by the government that have been delivered to the delivery destinations set forth in the…