15._DRAFT_Vance_FOS_QASP_(26_June_2013).pdf

DRAFT

FLIGHT OPERATING SUPPORT (FOS)

Quality Assurance Surveillance Plan (QASP)

Contract XXX

26 June 2013

ST

FLYING TRAINING WING (FTW), VANCE AFB OK

FLIGHT OPERATING SUPPORT QASP

71ST FLYING TRAINING WING, VANCE AFB OK

Chapter 1 – Purpose and Scope

1.1. Purpose 4

1.2. Scope 4

Chapter 2 – Performance Assessment Planning & Preparation

2.1. Planning and Preparation 4

2.2. Roles and Responsibilities 5

2.3 Required Training 11

Chapter 3 – Performance Assessment

3.1. Performance Assessment 14

3.2. Surveillance Methods 15

3.3. Other Surveillance Methods 17

3.4. Surveillance Schedule 18

3.5. Surveillance Assessment 19

3.6. Other Surveillance Assessment 19

3.7. Surveillance Ratings 20

3.8. Assessment of Contractor Quality Control Programs 21

3.9. Assessment of Non-SDS Items 22

3.10. Assessment of Service Provider Training Programs 22

3.11. Remedies for Unacceptable Performance 22

Chapter 4 – Performance Assessment Analysis

4.1. Data Review 23

4.2. Trend Analysis 23

Chapter 5 - Documentation of Service Provider Performance

5.1. Documenting Performance 23

5.2. Discrepancy Documentation 23

5.3. Repeat Write Ups 24

Chapter 6 – Performance Assessment Reporting

6.1. Scheduled Inspections and Observation Reports 24

6.2. Additional Reporting Requirements 24

6.3. Contractor Performance Assessment Reporting System (CPARS) 24

Chapter 7 – Performance Assessment Follow-Up

7.1. Follow-up Methods 25

71ST FLYING TRAINING WING, VANCE AFB OK

7.2. Performance Assessment Report (PAR) Follow-up 25

7.3. Corrective Action Report (CAR) Follow-up 25

7.4. Contractor Response 25

7.5. Final COR Recommendation 26

7.6. Disputes 27

Chapter 8 - Supporting Guidance and Documentation 26

Attachments

1. Aircraft Maintenance Surveillance Plan

2. Aircrew Flight Equipment Surveillance Plan

3. Airfield Management Surveillance Plan

71ST FLYING TRAINING WING, VANCE AFB OK

Chapter 1

Purpose and Scope

1.1. Purpose: This Quality Assurance Surveillance Plan (QASP) has been developed and will be implemented IAW FAR46.401, DFARS 246.401, PGI237.172, AFFARS MP5346.103 , AFFARS MP5301.602-2(d), Performance-Based Services Acquisition (PBSA) guidelines and

AFI 63-138. It is designed to provide an effective surveillance method of monitoring contractor performance for each listed objective on the Service Delivery Summary (SDS) in the contract as well as all contract requirements. This QASP provides a systematic method for observing and documenting the services that the service provider is required to furnish but not the details of how the service provider accomplishes the work.

1.2. Scope: This plan covers the Flight Operating Support (FOS) functional areas for the Vance

AFB FOS contract. Aircraft Maintenance, Aircrew Flight Equipment (AFE), and Airfield

Management are also covered more specifically as attached surveillance plans to this QASP developed under MP5346.103, MP5301.602, and AFI 21-101 and AETC Supplement 1 (Aircraft

Maintenance and Aircrew Flight Equipment).

1.2.1. This QASP is based on the premise that the service provider, not the government, is responsible for performance management and quality control actions to meet the terms of the contract.

Chapter 2

Performance Assessment Planning & Preparation

2.1. Planning and Preparation: The planning and preparation phase of contractor performance management should be a standard process conducted by the Functional Commander (FC), Chief

Contracting Officer Representatives (CORs) and CORs over the course of the contract execution.

There are several tools and resources available to the Government to aid in the management of the contractor’s performance. The first stop in the planning and preparation phase for the FC, Chief CORs and CORs should be the Performance Work Statement (PWS). This document will provide the guidelines of what services have been contracted by the Government and more importantly what levels of service are expected and acceptable. The guidelines by which to manage the execution of these services are outlined within this QASP. In addition to these documents, the FC, Chief CORs and CORs should familiarize themselves with the contractor’s policies and procedures in order to obtain a solid understanding of how the contractor manages their people and programs. The best way to obtain this information is through regular partnering meetings with the contractor.

2.1.1. Partnering: Partnering is the creation of a government/contractor relationship that promotes achievement of mutually beneficial goals during a government contract period or project. Partnering involves people who believe cooperation is the most effective and http://farsite.hill.af.mil/reghtml/regs/far2afmcfars/af_afmc/affars/MP5346.103.htm http://farsite.hill.af.mil/reghtml/regs/FAR2AFMCFARS/AF_AFMC/AFFARS/MP5346.103.htm?zoom_highlight=mp5301%2E602-2+d

71ST FLYING TRAINING WING, VANCE AFB OK

efficient way to achieve their goals. The ultimate goal of partnering is to change the “us vs.

them” attitude and create the “we” relationship.

2.1.2. The most important step in partnering is for each party to first seek to understand the goals, objectives, and interests of the other party. This will create a “win-win” situation.

Efficient partnering has specific attributes that contribute equally to its success. They are common goals, communication, problem solving, and synergies. These attributes seek to accomplish the ultimate goal of cooperation and shared vision.

2.1.3. The Defense Acquisition University (DAU) website offers a Continuous Learning

Course (CLC045) called Partnering. It is highly recommended that each FC and COR at least browse through this course for a better understanding of the concept of partnering and its effects on working relationships.

2.2. Roles and Responsibilities:

2.2.1. Functional Commander (FC). The FC is the 71th Operations Group Commander.

The FC is the government’s functional authority for the contracted function. The FC retains all responsibility for the success or failure of the contracted function, the same as if the contracted function was a governmental activity. The FC is responsible for assessing the contractor’s performance and managing the requirements over the life of the contract per

MP5346.103 and MP5301.602. Specifically, the FC shall:

a. Receive training within 30 days of commencement of duties IAW AFFARS

MP5346.103. This training is conducted by the QAPC and will be tailored to the FC needs.

b. Review all pertinent documents to include: the Performance Work Statement (PWS), the contractor’s proposal, the QASP, service provider quality documents, their training plan, required regulations/publications, service provider and government strike plan annually, and their area’s CPAR report.

c. Communicate expectations to the service provider’s Program Manager or area director as applicable.

d. Work with the ACO, through 71 FTW/QAV, in developing or modifying procedures in order to meet governmental expectations and contract standards.

e. Supervise and oversee COR personnel and functions.

2.2.2. Quality Assurance Evaluation and Standardization (QAV). QAV serves as the

Program Management Office responsible for enforcing all policies regarding management, execution, and oversight of the contract as delegated by 71 FTW/CC. Additionally, 71

FTW/QAV is responsible for ensuring that contract surveillance, evaluation and reporting practices are standardized and enforceable across all areas of the Vance FOS Contract. 71

FTW/QAV serves as the action office for all contract compliance and service delivery issues that cannot be resolved at the FC or Group level.

2.2.3. Program Manager. The Chief of QAV has been designated as the Government

Program Manager (PM) for the Vance FOS Contract by the 71 FTW/CC. The PM is http://www.dau.mil/Training/default.aspx

71ST FLYING TRAINING WING, VANCE AFB OK

responsible for management oversight of all contracted activities covered under contract #.

Specifically, the PM shall be responsible for monitoring cost, schedule, technical performance, and risk assessment/mitigation on the contract as delegated by 71 FTW/CC.

2.2.3.1. Contractor Interface: The PM shall serve as the single point of contact between the Contractor PM and the Government. Any issues requiring wing leadership involvement shall first be presented to the Government Program Manager for disposition.

However, contract interpretation can only be done by a warranted contracting officer.

2.2.3.2. Technical Performance: The PM shall oversee technical performance of the contract IAW the PWS. The PM, with the Administrative Contracting Officer, (ACO), shall serve as an advisor to the FC with respect to acceptable levels of performance by the contractor. Unacceptable levels of performance by the contractor shall be handled IAW this QASP.

2.2.3.3. Financial Management: The PM shall be responsible for assessing the contractors’ financial execution of the contract and reporting this information to wing leadership on an as needed basis.

2.2.3.4. Risk Assessment: The Program Manager shall work with the FC to assess overall program risks for service delivery summary (SDS) items not being met. Once a risk assessment has been made, the PM shall work with the FC and ACO to establish an appropriate risk mitigation plan to alleviate the non-conformance. NOTE: The FC retains all responsibility for the success or failure of the contracted function, the same as if the contracted function was a governmental activity. Issues regarding contract requirements not being met should be addressed between the Chief COR, FC and the Contractors’ director overseeing the activity in question prior to escalation to the PM. However, when circumstances arise that cannot be worked at the FC level, the FC should notify the PM and ACO for determination/assistance.

2.2.4. Chief Installation Contracting Office (ICO). Ensures a Quality Assurance Program

Coordinator (QAPC) is appointed to integrate the quality contract requirements into the quality assurance program. The Chief, ICO shall ensure functional area leadership receives tailored training for PBSA and Quality Assurance (QA). The Chief, ICO shall ensure QAPC individual training plans are tailored to the local mission and local QAPC responsibilities.

The Chief, ICO shall ensure consistent guidance to customers is given by the QAPC and other functional areas such as the Acquisition Centers of Excellence or similar offices.

2.2.5. Administrative Contracting Officer (ACO). Appointed (warranted) government agents authorized to award or administer contracts. The ACO is the only person authorized to contractually obligate the government or direct the contractor. The ACO delegates quality assurance responsibilities to CORs, giving them the authority to formally evaluate and accept contractor services.

2.2.5.1. Contract Interpretation: The ACO is responsible for contract interpretation and shall serve as the advisor for developing incentives/remedies as appropriate tied to performance objectives and performance thresholds set forth in the contract. The ACO

71ST FLYING TRAINING WING, VANCE AFB OK

shall have final decision authority regarding contract language disputes with the

Contractor.

2.2.5.2. Contract Administration: The ACO shall be responsible for overall contract administration overseeing contract modifications and contract file documentation. The

ACO shall ensure that all contract documentation is maintained IAW applicable governing directives and AFIs.

2.2.5.3. Quality Assurance: The ACO is responsible for ensuring appropriate quality assurance has been conducted before accepting services under the contract.

2.2.5.4. Training: The ACO shall be responsible for ensuring functional area leadership receives tailored training for PBSA and Quality Assurance. The ACO is responsible for the content of all contract-specific training, including Phase II training.

2.2.6. Quality Assurance Program Coordinator (QAPC). Base-level individual, normally from the contracting activity, selected to coordinate the Quality Assurance program. The

QAPC is responsible for training the functional commanders and CORs on surveillance methods and overseeing the effectiveness of the quality assurance surveillance program. The

QAPC shall be responsible for all requirements outlined in AFFARS MP5346.103.

2.2.7. 71 FTW Chief Contracting Office Representative (COR). Develops, updates, implements, and executes plans and programs to provide for the standardization and evaluation of 71 FTW Quality Assurance Program and contract compliance efforts. Works with CORs, the ACO, and FC to identify, document, and troubleshoot contract compliance issues. Standardizes and evaluates Contractor Performance Assessment Reporting System

(CPARS) inputs from COR’s and FC. The 71 FTW Chief COR will perform over the shoulder inspections with each Chief COR and individual COR’s as required. The 71 FTW

Chief COR will also be responsible for ensuring surveillance of PWS Section 2 for each department is accomplished. The checklists used for these inspections can be located on the s: drive/public/QAV/checklists. The Wing Chief COR shall assist as needed, the QAPC to ensure that each COR is initially evaluated to determine qualifications, experience and ability to accomplish technical inspections and contract surveillance.

2.2.8. Aircraft Maintenance, Aircrew Flight Equipment, and Airfield Management Chief

Contracting Office Representative (COR). Responsible for ensuring surveillance and documentation of contract performance and report noncompliance or abnormalities to the

Functional Commander(s), the Wing Chief COR, and the ACO. Specifically, the Chief COR shall:

a. Verify that the service provider meets contract obligations as specified in the contract.

b. Review/document QASP and checklists as required for adequacy. Submits updated

QASP and checklists to QAV and QAPC as changes occur.

c. Review the service provider’s quality control program for acceptable quality level and recommends acceptability to the ACO through the Functional Commanders.

http://farsite.hill.af.mil/reghtml/regs/far2afmcfars/af_afmc/affars/MP5346.103.htm#TopOfPage file:///S:/Public/QAV/Checklists

71ST FLYING TRAINING WING, VANCE AFB OK

d. Ensure each COR is initially evaluated to determine qualifications, experience, and ability to accomplish scheduled inspections and contract surveillance functions. It is the responsibility of the Chief COR to coordinate with QAPC to ensure all required training, such as DAU Course COR 222 (COR Training) and CLM 003 (Ethics), and

Phase I & II, is completed by all new CORs. Also ensures refresher training for these courses is attended annually. Chief COR must attend AETC Chief COR Training

(Block 3) either at AETC or locally. All required training shall be scheduled and coordinated through the QAPC.

e. Ensure that each COR is performing their duties IAW this QASP and MP5346.103 and MP5301.602.

f. Assist the ACO in managing the applicable government-furnished property clause of the contract.

g. Ensure a monthly summary of all COR surveillance activities is reviewed and signed by the Functional Commander(s) and is provided through the QAV office, to the

ACO, no later than the 5 th workday of the month. The Chief COR shall mark whether performance was satisfactory or unsatisfactory for the month.

h. Review of contractor proposals. All facets of the proposal, within the technical expertise of CORs, will be evaluated to include number of personnel, skill level of personnel, man-hours proposed, and all associated costs.

i. Assist HQ AETC/A4PM, Functional Commander(s), QAV, and/or the ACO in determining contract cost estimates, if requested. Also creates modifications to the contract IAW the Modification Process Policy attached to this QASP. Use 71 FTW

Modification Tracker Database to update status of process.

j. (Reserved)

k. Perform surveillance activities as required.

l. Review and evaluate service provider-submitted value engineering change proposals.

m. Provide assistance to the Wing Safety Office, or equivalent, in mishap and incident reporting.

n. Review service provider regulations and/or instructions prior to acceptance and publication.

o. Verify the statistical information provided by the service provider that concerns the standards specified in the Performance Work Statement (PWS) or appendix of the contract.

p. Gathers past performance information and provides to ACO through QAV.

71ST FLYING TRAINING WING, VANCE AFB OK

q. Attend monthly QAV/Chief COR meetings. If unable to attend, an alternate shall be assigned.

r. Review service provider and governmental strike plan annually. Submit updated/revised governmental strike plan as required to the ACO.

Aircraft Maintenance

Additional chief COR responsibilities are outlined in AFI 21-101, AETC SUP, Chap 18.

Superintendent Responsibilities. The Superintendent COR is directly responsible to the

Chief COR to ensure the complete implementation of AFI 21-101 & AETC SUP, Chap

18, and this surveillance plan. In addition the Superintendent will:

a. Assist the Chief COR with administration of the contract award fee program and validation of all associated data.

b. Perform the duties of the Chief COR in his/her absence.

c. Assist the Chief COR, FC, and ACO with clarification of issues between CORs and the service provider concerning the validity of documented discrepancies.

Lead Inspector Responsibilities. The lead Inspector is directly responsible to the Chief

COR to assist with the complete implementation of AFI 21-101 & AETC SUP, and this surveillance plan. In addition the Lead Inspector will:

a. Focus COR efforts and standardize surveillance and documentation methods.

b. Evaluate COR qualifications through Technical Inspections and Area

Observation Inspections, to ensure standardized practices, and identify to the

Chief COR individuals for specialty and additional proficiency training

c. Perform the duties of the Superintendant or Chief COR in his/her absence.

d. Manage Training Program.

e. Review COR discrepancies contained in QMIS to assist Chief COR in effective surveillance.

2.2.9. Contracting Office Representative (COR). The CORs primary purpose is to surveil and document. CORs are the “eyes and ears” of the Wing Commander, Functional

Commander(s), Program Manager, and ACO relative to the actual application of the contract;

however, they are not a quality control function. Another important part of the CORs responsibilities is to “partner” with the contractor regularly to ensure open communications between the government and contractor. The COR is to be objective, fair, and consistent in evaluating service provider’s performance, against scheduled and regulatory requirements and the terms of the contract. CORs will not direct work or re-accomplishment of work, 71ST FLYING TRAINING WING, VANCE AFB OK change the contract, or formally interpret the contract. The ACO resolves these types of issues. CORs are prohibited from training or advising the contractor and/or performing contracted tasks or quality control work. However, other government employees may do this under certain circumstances. If requested by the contractor, the contracting officer, under the partnering principle, may direct other government employees to train or advise the contractor to help them get to a level of competence required by contract standards or in support of new equipment the government has provided them. Specifically, the COR shall:

a. Know the specifications and requirements of the contract.

b. Know and maintain proficiency in contract surveillance procedures and requirements.

CORs will maintain technical competency in their functional areas, maintain proficiency in contract surveillance techniques, and assist with formulating the PWS and observation area inspection guides. CORs will review/approve contractor-developed instructions noting necessary changes/updates in Quality Management

Information System (QMIS) prior to signature and implementation as directed by the

Chief COR. QMIS is referred to as the “SMART” system in PWS Section 2. CORs will accomplish contract surveillance by evaluating and documenting service provider performance and inform the Chief COR and FC /FD when service provider performance does not meet performance thresholds.

c. Know and apply the procedures for documenting surveillance.

d. Perform surveillance according to their surveillance schedules.

e. Maintain scheduled competency in his/her assigned surveillance areas.

f. Attain qualification in the appropriate areas before performing evaluations, inspections, or surveillance duties unsupervised.

g. Review the deficiency and mishap service provider reports for accuracy, adverse trends, and mission accomplishment. Additionally, reviews reports to higher headquarters for possible indicators of performance trends.

h. Serve as a member of the source selection team when selected.

i. Develop surveillance schedules.

j. Review service provider regulations and/or instructions prior to acceptance and publication.

k. Be familiar with emergency procedures to be implemented if default or strike interrupts service provider performance.

l. Ensure service provider does not answer higher headquarter suspenses and requests without prior approval from local leadership.

71ST FLYING TRAINING WING, VANCE AFB OK

m. Use proper governmental and contractor chain of command when elevating issues to upper management.

COR responsibilities are identified in AFI 21-101 & AETC SUP, and this surveillance plan

2.2.10. Reserved

2.2.11. Performance Management Council (PMC). The purpose of the Performance

Management Council is for the government and the service provider to collectively review the service provider’s performance on a quarterly basis. Areas addressed are:

SDS performance, surveillance results, contract administration and current issues. The areas addressed may change depending on current leadership concerns/issues. The slides for this meeting are created by the Chief COR and are partnered with the FC and their contractor counterpart prior to submission to QAV. Slides are submitted to QAV on the th business day of the month.

2.2.11.1. The meetings are scheduled on the wing’s master meeting schedule and therefore all responsible parties are aware of their frequency. QAV may send courtesy meeting reminders to council members.

2.2.11.2. Members of the PMC include, but are not limited to: the service provider, the Wing CC, Group CCs, QAV, the ACO, FCs, and Chief CORs.

2.3. Required Training. All CORs will complete the following Defense Acquisition University

(DAU) training courses:

1. DAU COR 222, Contracting Officer’s Representative Course

2. DAU COR 206, COR in a Contingency Environment (when applicable)

3. CLM 003, Ethics Training for Acquisition Technology and Logistics

4. Joint Ethic Regulation DoD 5500.7-R (review/understanding of regulation)

5. Additional Training as required by CO and

6. Combat Trafficking In Persons

Training will be conducted IAW the COR training plan and consist of the mandatory requirements identified in AFI 21-101 & AETC SUP, and requirements locally determined by the FC, Chief COR, and Superintendent. Individual training records will be maintained

IAW AFI 21-101 & AETC SUP, and AFI 36-2201, Air Force On-The-Job Training

Administration.

71ST FLYING TRAINING WING, VANCE AFB OK

a. Newly assigned CORs must complete Phase I and II training prior to performing surveillance duties.

b. To ensure adequate coverage, CORs are also trained in areas of responsibility and duties outside their normal AFSC by Cross Utilization Training (CUT). AF/AETC directives and technical orders will be used to complete On the Job Training (OJT).

Training will be documented in the Training Business Area (TBA).

c. All CORs, Superintendent, and Chief COR will complete the periodic refresher and recurring training prescribed by AFI 21-101 & AETC SUP. The FC /Chief COR will project required TDY funding in the unit budget.

d. The Chief COR ensures personnel are available or on call during all contractor activities. Weekend security check will be covered by one COR, one weekend per month. Work schedules/duty hours for the COR workforce (military and civilian) will be developed and posted in the COR Section. Work schedules will ensure surveillance coverage of all service provider performance periods randomly throughout each month.

Aircrew Flight Equipment

a. Initial contract surveillance training. Formal training conducted in two phases.

Completed prior to performing surveillance duties:

1) Phase I, General QAE training provided by the Quality Assurance Program

Coordinator (QAPC) (Note: AETC QAE Course, 393AET0066-002, fulfills the above requirement.) AETC requires, Phase I training to be successfully completed within 90 days of assignment. (COR’s require course 393AET0066-

002 refresher every 3-years.)

2) Phase II, Contract Specific Training. (Note: This training is provided by the

Administrative Contracting Office) To stay current on the PWS and surveillance plan, refresher Phase II Contract Specific Training refresher training must be completed annually.

b. COR Training Certifications: COR’s will be knowledgeable of the task they surveil.

They are duty positioned qualified to inspect, surveil, and observe according to AFI’s

11-301 and other applicable directives.

c. COR Training records: All training will be maintained IAW AFI 36-2201 and the

1POX1 Career Field Education and Training Plan (CFETP). COR’s will maintain an AF

IMT 797 that identifies specific responsibilities listed in AFI 21-101 AETC Sup-1.

Training records will be filed in the COR work-center.

2.4 Combating Trafficking in Persons. DoD has a zero tolerance policy for human trafficking.

CORs are the first line of defense in the battle against human trafficking and must complete

71ST FLYING TRAINING WING, VANCE AFB OK

Combating Trafficking in Persons training annually. As the COR monitors the contractor and its employees, Combating Trafficking in Persons should rank among the COR’s chief priorities.

CORs must be diligent in ensuring that contractors and contractor personnel are not trafficking in persons. During surveillance, the COR will monitor the contractor’s performance regarding trafficking in persons for compliance with all that is required IAW FAR clause 52.222-50, Combating Trafficking in Persons. The COR must inform the Contracting Officer if the contractor, contractor personnel, subcontractor, or subcontractor personnel have failed to comply with the requirements of the clause at FAR 52.222-50. After receiving this information, the

Contracting Officer shall provide information for any investigation and enforcement to:

Program Manager

DoD CTIP Law Enforcement and Support

OUSD (P&R) DHRA

4800 Mark Center Dr Suite 06J25-01

Alexandria, VA 22350-4000

Reports may also be made:

On-line at http://ctip.defense.gov/;

On-line via the National CTIP Hotline website at http://www.polarisproject.org/;

By e-mail to CTIPReports@OSD.Pentagon.mil; or

By phone to the National CTIP Hotline at 1-888-373-7888.

Note: The COR must report any suspected violations or activities to the Contracting Officer

(See FAR Subpart 22.17 and DFARS Subpart 222.17).

Note: CORs should not personally investigate suspected incidents of Trafficking in Persons but should forward all reported or suspected violations to the Contracting Officer immediately.

Note: The Contractor must take appropriate action should a violation occur. For corrective procedures and remedies for non-compliance, see FAR 52.222-50. ”

2.5 Value Engineering Proposals. This program is an incentive plan that encourages civilian maintenance contractors to develop and engineer new and innovative ways to save the government money. By submitting cost-reduction methods to the government under the value engineering program, contractors can help the government cut costs and, at the same time, share in the savings. See AFI 63-101, Acquisition and Sustainment Life Cycle Management, and maintenance contracts for submission criteria and procedures for value engineering change proposals.

71ST FLYING TRAINING WING, VANCE AFB OK

Chapter 3

Performance Assessment

3.1. Performance Assessment. Performance assessment is evaluating, assessing, and documenting the contractor’s performance IAW the overall QASP. The premise behind performance assessment is that the contractor, not the government, is responsible for managing and ensuring that performance meets the terms of the contract. It is the government’s responsibility to monitor performance and hold the contractor accountable for performance. All areas of the contract shall be surveilled and the performance of the contractor shall be assessed and reported in accordance with this QASP.

3.1.1. Initial Contract Performance Review. The initial evaluation of contractor performance is a joint determination by the Multi-Functional Team (MFT) that the contractor has successfully started performance, completed transition, is fully operational, and is within the estimated cost, schedule, and performance parameters of the contract. IAW AFI 63-138, para 6.4, the Services Designated Official (SDO) may waive the initial evaluation for contractors that have continued performance under a successor contract award (prior incumbent), or for contractors that have otherwise demonstrated full compliance with contract start-up. Separate initial contract performance reviews are not required when they would duplicate existing reviews.

3.1.2. All functional areas in the FOS are required to also ensure surveillance of PWS

Section 2 which is applicable to all contractors. The following table shows the Performance

Thresholds and frequency of surveillance for PWS Section 2.

ITEM

PERFORMANCE

OBJECTIVE

REFERENCE

PARAGRAPHS

PERFORMANCE

THRESHOLD

METHOD OF

SURVEILLANCE

FREQUENCY

OF

INSPECTIONS

A Fully trained, qualified, and knowledgeable employees who are customer service oriented

2.1.3 - Personnel 100% of employees declared position qualified are fully trained, task proficient, and task certified.

Periodic Inspections and Random

Sampling

Monthly

B Effective quality control program

2.1.4 – Quality

Control

Continuous quality of services, workmanship, and products.

Periodic Inspections and Random

Sampling Monthly

2.3.5.4 –

Government

Observations

Periodic Inspections and Random

Sampling

Monthly

C Effective occupational health and safety program

2.1.5 –

Occupational Health and Safety

Continuous protection of AF resources (GOCO facilities to include hangars and industrial areas, equipment, and AF

Periodic Inspections and Random

Sampling

Monthly

71ST FLYING TRAINING WING, VANCE AFB OK

personnel)

D Effective security program

2.1.6 – Security

Requirements

100% compliance with all Government security requirements

Periodic Inspections and Random

Sampling Monthly

E Effective environmental protection program

2.1.9 –

Environmental

Protection (see

Section I)

100% compliance with all environmental protection requirements

Sampling

Quarterly

F Effective emergency plans & support service program

2.1.10 –

Emergency Plans &

Services

100% support of emergency plans and services – No less-than-satisfactory evaluation of emergency plans or emergency support service

Sampling Quarterly

G Effective use of technical guidance

2.1.11 –

Technical Guidance

100% usage of technical guidance in the performance of maintenance and technical services were technical guidance is required

Sampling

Aircraft Maintenance (PWS Section 3)

Attachment 1

Aircrew Flight Equipment (PWS Section 6)

Attachment 2

Airfield Management (PWS Section 21)

Attachment 3

3.2. Surveillance Methods. The COR will ensure contract compliance through various surveillance methods. These methods may include but are not limited to: periodic inspections, random sampling, customer feedbacks/complaints, service provider metrics, 100% surveillance and quality indices third party audits.

3.2.1. Periodic Inspections: These items are inspected using periodic surveillance (daily, weekly, monthly, quarterly, etc.) as determined by the COR. The results of the periodic

71ST FLYING TRAINING WING, VANCE AFB OK

surveillance inspections may be used as the basis for actions (other than payment deductions) towards the contractor. In such cases, the Inspection of Services clause becomes the basis for the contracting officer’s actions. Periodic inspections may be conducted via over the shoulder inspections, spot sampling, or review of the contractor reports and other deliverable documents.

3.2.2. Random Sampling: Random sampling is an appropriate method for frequently recurring tasks. A typical use of this surveillance method is lot size sampling. Initially, the

COR will evaluate randomly selected samples of the activity to determine the acceptability of the entire requirement. This method reduces the time the COR must spend on surveillance duties, and yet still gives a reasonably accurate picture of a service’s overall acceptability.

3.2.3. Customer Feedbacks/Complaints: Customer inputs will be registered in QMIS allowing FC to review as required. The Chief COR, or designated COR, should review all customer feedbacks.

3.2.3.1. When a Customer Feedback Service Delivery Summary (SDS) is present, the

COR must collect and review the feedbacks prior to the service provider unless the feedbacks are directly entered into QMIS by the customer. This may be accomplished by placing a lock box in each facility that offers hard-copy customer feedbacks. The COR will maintain the key to this lock box. As a courtesy, before removing the customer feedbacks, the COR will notify the service provider that he/she is removing them. After

COR review, the feedbacks will go to the service provider for review/action. The COR should review and return the feedbacks to the service provider within 24 hours of receipt.

If the service provider has a contractual requirement to answer negative feedbacks within a certain amount of time, that time will not begin until the service provider receives the feedback from the COR.

3.2.3.2. If there is a requirement for negative feedback follow-up, the COR must monitor that follow-up. The COR will validate the actual occurrence and/or the results if the situation requires further investigation. The Chief COR will consider the actions taken by the service provider and will determine if the response is expected to resolve the customer’s complaint. If the action fails to address the problem or is inadequate, the

Chief COR and FC shall contact the ACO through QAV for further action.

3.2.3.3. If an SDS requiring a performance threshold on customer feedbacks is in the

PWS and minimal feedbacks are received, on a monthly basis, each COR will contact government customers for their feedback on the service providers’ performance. The number of feedbacks received will be determined by the need in each department.

3.2.4. Service Provided Metrics: Each department or functional area has Service Delivery

Summaries (SDSs) which are part of the PWS. They are the major areas of concern for each functional area. However, these are not the only requirements that are measured. Every

“shall” statement within the PWS should be measured. This includes Section 2 of the PWS which is applicable to all areas. SDSs are found in a table at the end of each PWS section.

They are broken down by Performance Objective, PWS Reference, and Performance

71ST FLYING TRAINING WING, VANCE AFB OK

Thresholds. This table details what and how often each requirement is inspected or surveilled. Changes to SDSs can only occur with a modification to the contract.

3.2.5. One-Hundred Percent Inspection: When this type of surveillance is used, the COR must inspect and evaluate the contractor’s performance each time it is performed. The result of the contractor’s overall performance is then evaluated to determine acceptability of the lot.

3.2.6. Third Party Audits or Assessments (when appropriate): Third party audits or assessments refer to contractor evaluation by a third party organization that is independent of the government and the contractor. All documentation supplied to, and produced by, the third party should be made available to the government by the contractor.

3.3. Other Surveillance Methods. The following are types of surveillances that may be used throughout the contract period to evaluate service provider performance when structured surveillances do not exist.

3.3.1. Budget Monitoring: The Chief COR will monitor service provider’s compliance and advise him/her of budgetary trends when needed. CORs will surveil supply listings to ensure supplies ordered are those needed to perform the mission and that funds are provided for these purchases. The COR will ensure the service provider includes the COR on routing for all purchase requests. The COR will ensure service provider includes governing reference for authorization of purchases on all purchase requests. Financial plans and unfunded requirements developed by the service provider will be surveilled by the COR. The COR will be included on these routings as well as all purchase requests.

3.3.2. Exercise/Contingency Surveillance. The service provider’s participation in exercises/contingencies will be surveilled by a COR assigned to the Wing Exercise

Evaluation Team (WEET). CORs may be appointed to the WEET. As a minimum, service area related exercises will be surveilled using appropriate checklists.

3.3.3. PWS Section 2 Surveillance. This section should be surveilled at least annually by each functional area. The FC and Chief COR will determine the frequency of these inspections. Inspections of Section 2 will be documented in QMIS just as all other inspections are documented. These inspections can be conducted with other inspections but should include verbiage of what specific requirements from Section 2 were covered during the inspection.

3.3.3.4. Cost Reimbursable Items. The Chief COR will validate the accuracy of contractor submitted financial figures prior to the government paying for services by coordination with the Resource Advisor (RA) COR and other CORs. CORs will conduct appropriate research/validation for all contractor claims/invoices/vouchers and travel expenses.

3.3.3.4.1. Monthly Payment Invoices. The COR will ensure costs are validated in coordination with the ACO, ensure proper Contract Line Item Number (CLIN) assignment, proper Cost Center, Funds Cite and Accounting Classification Reference

Numbers (ACRN) are used.

71ST FLYING TRAINING WING, VANCE AFB OK

3.3.3.4.2. The RA COR will monitor the service provider’s Micro-purchase summaries during the RA Observation guide. The RA will notify the Chief COR of any abnormal expenditures or cost trends by the contractor. Upon notification, CORs will investigate to ensure proper ordering procedures are followed.

3.3.3.5. Contractor Proposals. The Chief COR along with the ACO will review and provide surveillance of contractor proposals. All facets of the proposal, within the technical expertise of COR, will be evaluated to include number of personnel, skill level of personnel, and man-hours proposed and all associated costs.

3.4. Surveillance Schedule. A Chief COR will develop a schedule of surveillance based on the requirements of FAR Part 46, MP5346.103 and MP5301.602, and this QASP. This schedule shall include surveillance of contractor performance as appropriate, outside of the CORs normal duty hours at least monthly (i.e., all shifts as required). Surveillance schedules should specify all work requiring surveillance and should specify the method of surveillance. They should be prepared in conjunction with the preparation of the statement of work. The surveillance schedule will be completed no later than seven calendar days prior to the beginning of the period it covers, and a copy will be sent to the ACO and QAV. The FC should sign the schedule prior to submitting. The schedule is “FOR OFFICIAL USE ONLY” and is not releasable to anyone other than authorized government personnel.

3.4.1. Surveillance Schedule Changes. The Chief COR will forward changes to monthly surveillance schedules to the ACO and the Wing Chief COR, as changes occur.

3.4.2. Determining surveillance frequency. During performance assessment planning, the

Chief COR shall consider using operational risk, service complexity and criticality as factors in deciding the performance assessment plan from month to month. If a particular function of the contractor’s performance has a continuing record of acceptable performance, and unacceptable performance would not likely result in loss of life to AF personnel or damage to government property, surveillance of that function may be reduced. If contractor performance of a function is less than satisfactory, surveillance of that function should be increased. When this is determined to be appropriate, the Chief COR, with FC and

PCO/ACO approval, will adjust the surveillance schedule.

3.4.3. Monthly Surveillance Shortfall. If minimum monthly surveillance requirements cannot be accomplished due to equipment non-availability or special circumstances, the reason(s) for missing inspection will be documented on the end of the month’s summary report.

3.4.4. Waiver of scheduled observation area inspections. If the Chief COR determines a contractor work center has a continuing record of acceptable performance, its scheduled observation area inspection may be waived for one observation period (month, quarter, etc.), if approved by the FC and the ACO. The Chief COR will include comments concerning the waived observation area inspection in the monthly summary. Surveillance will never be waived for more than one prescribed period and the surveillance frequency will go back to normal if contractor performance shows signs of less than acceptable performance.

71ST FLYING TRAINING WING, VANCE AFB OK

3.4.5. The Chief COR will monitor surveillance results to ensure no particular work areas/sites and/or office areas are being disproportionately inspected. The Chief COR may, however, deviate from random selection to ensure a broad cross-section of the service area is being inspected to investigate potential problems or if trends indicate a need. It is essential that CORs accomplish sufficient in-depth inspections in all areas to measure the quality of service provider performance and provide an effective measurement/assessment to ensure overall performance meets contract requirements. Compliance with applicable directives prescribed by the PWS, service provider-developed Quality Control Program (QCP), Contract Sections, and plans/instructions will be part of the surveillance.

3.5. Surveillance Assessment. It is essential that CORs accomplish sufficient in-depth inspections in all areas to measure the quality of service provider performance and provide an effective measurement/assessment to ensure overall performance meets contract requirements.

Compliance with applicable directives prescribed by the PWS, service provider-developed

Quality Assurance Program, Contract Sections, and plans/instructions will be part of the surveillance. The contractor’s maintenance data will be reviewed for accuracy. Service provider performance and mission reports, higher headquarters logistic reports, and AETC maintenance summary will be reviewed for possible indications of negative trends.

3.6. Other Surveillance Assessments. In addition to Technical and Observation area inspections, performance assessment includes the following:

3.6.1. As Observed Deficiencies. These deficiencies will be documented on a PAR. These reports will not be classified as conform/non-conform and discrepancies will not be categorized as major/minor. They occur when discrepancies or deficiencies are observed or discovered that are not directly associated with a scheduled inspection. Monthly and quarterly Service Delivery Summary infractions will be documented as an As Observed finding until further action is required.

3.6.2. Safety Violations. These violations can be documented as part of the inspection being performed, or if appropriate, "as observed" provided the violation clearly indicates the potential to damage government resources. CORs do not document violations of OSHA or

Air Force safety standards that do not present the potential to damage/injure government resources; these violations will be reported via email to the ACO.

3.6.3. Exercise Contingency Surveillance. The contractor’s participation in exercises and contingencies will be evaluated by COR(s) assigned to the Exercise Evaluation Team (EET).

Findings will be reported to the Chief of the Exercise Evaluation Team and documented “as observed” if deemed appropriate.

3.6.4. Repeat Findings. A repeat finding is defined as a duplicate finding in the same aircraft maintenance section within one quarter or the next scheduled area observation. When documenting repeats, a minor repeat finding will be documented as a minor but will have the word REPEAT documented in the finding in QMIS. If a repeat occurs a second time on the next consecutive inspection, it will be documented as a major finding and the issue will be

71ST FLYING TRAINING WING, VANCE AFB OK

brought to the ACO. It is the responsibility of the COR inspector to review QMIS and track repeat occurrences in assigned areas.

3.6.5. Follow-up Inspections. CORs will verify that all findings are complied with prior to

COR close out in QMIS. If a follow-up inspection is accomplished and produces a finding in between scheduled inspections, they will be treated as an As Observed. An As Observed will also be documented if a finding is not corrected and documented complete by the service provider or the estimated completion date in QMIS has expired.

3.6.7. Government Furnished Equipment/Property Surveillance (GFE/P). All CORs will provide surveillance in the management of GFE/P during routine performance of work center observation guides. Service provider work center GFE/P will be checked for proper item identification/labeling and identification on the master inventory located on the MA server.

Surveillance will include checks for evidence of fraud, waste or abuse. The Government property inventory conducted at the beginning of the contract is maintained in the ACO office.

3.6.7.1. The GFE/P Monitor COR will monitor/surveil the service provider GFE/P management program to include processes for purchase, control and disposal of GFE/P using a locally developed observation area guide. The GFE/P Monitor COR will validate inventory updates provided by the service provider, maintain a copy of the validation sheets and report the results of the validation via a memorandum to the ACO.

3.6.8. Customer Complaint. Inputs for the FOS contract will be registered in QMIS. The

Chief COR, Superintendent or designated COR will review all customer complaints. The

COR will validate the actual occurrence and/or the results if the situation is deemed serious enough for further investigation. The Chief COR or Superintendent will consider the actions taken by the service provider and if the response is expected to solve the customer’s complaint. If the action fails to address the problem or is inadequate, the Chief COR or

Superintendent will contact the ACO for further actions. Feedbacks will be tracked and documented on a locally produced spreadsheet to be discussed during partnering meetings with the service provider.

3.7. Surveillance Ratings: A discrepancy is any validated inspection finding or observation.

Each FOS FC and Chief COR will determine then partner with the service provider on what constitutes a major versus a minor discrepancy for their areas. Many of the Service Delivery

Summary (SDS) items in our PWS describe the type of discrepancies that constitute a failure to comply with contractual requirements. Any discrepancy already defined in the PWS as a failure is considered a major discrepancy. Additional major discrepancies will be described by the FC and Chief COR, but are primarily defined as any contractor actions that could result in a violation of safety, security, protection of resources, or law.

3.7.1. Major deficiencies include but are not limited to: A step serious enough to adversely affect the performance of the equipment involved is omitted or improperly completed;

violation of federal, state or local laws; DoD and Air Force environmental protection policies and directives; TO violations, to include not using required TOs to perform tasks, violations

71ST FLYING TRAINING WING, VANCE AFB OK

of mandatory directives and supplements when such may cause damage to government property or injury to government personnel; improper use of tools or use of outdated data when such may cause damage to government property or injury to government personnel; or accomplishing procedures without training or qualification on the task.

3.7.2. A minor discrepancy is any failure to comply with the contract that is not already identified as a major, does not compromise Vance’s mission, and is easily correctable.

3.7.3. Baselines for number of minors that constitute a major are determined by the FC and

Chief COR. Minor discrepancies that consist of a grouping of like deficiencies may be documented as one or several discrepancies. When grouping deficiencies, CORs will ensure surveillance techniques are applied consistently across inspection areas. Surveillance inspections will be rated as either acceptable or unacceptable.

3.7.4. "As Observed" inspections are unscheduled inspections. They occur when discrepancies or deficiencies are observed or discovered that are not directed and reported.

“As Observed” inspections are rated as conform and non-conform. “As Observed” inspections should be documented as a PAR or CAR as deemed appropriate by the FC.

These findings, however, will not count against the contractor’s findings rate. If “As

Observed” findings highlight a potential performance issue, consideration should be given to adding the task or tasks to the COR’s regularly scheduled inspections. Once included in the regular inspections, any further discrepancies shall be noted and will count against the contractor’s finding’s rate.

3.8. Assessment of Contractor Quality Control Programs: All areas of the contract discuss the contractor’s quality control but PWS Section 2 is most specific.