Attach 4 - QASP_Vance BOS 21 FEB 2020.pdf

VANCE AFB

BASE OPERATING SUPPORT (BOS)

Quality Assurance Surveillance Plan (QASP)

Contract FA300220R0007

10 FEB 2020

ST

FLYING TRAINING WING (FTW), VANCE AFB OK

BASE OPERATING SUPPORT QASP

71ST FLYING TRAINING WING, VANCE AFB OK

TABLE OF CONTENTS

Chapter 1 – Purpose and Scope

1.1. Purpose

1.2. Scope

Chapter 2 – Performance Assessment Planning & Preparation

2.1. Planning and Preparation

2.2. Roles and Responsibilities

2.3 Required Training

Chapter 3 – Performance Assessment

3.1. Performance Assessment

3.2. Surveillance Methods

3.3. Other Surveillance Methods

3.4. Surveillance Schedule

3.5. Surveillance Ratings

3.6. Assessment of Contractor Quality Control Programs

3.7. Assessment of Non-SS Items

3.8. Assessment of Service Provider Training Programs

3.9. Remedies for Unacceptable Performance

Chapter 4 – Performance Assessment Analysis

4.1. Data Review

4.2. Trend Analysis

Chapter 5 - Documentation of Service Provider Performance

5.1. Documenting Performance

5.2. Discrepancy Documentation

5.3. Repeat Write Up

Chapter 6 – Performance Assessment Reporting

6.1. Scheduled Inspections and Observation Reports

6.2. Additional Reporting Requirements

6.3. Contractor Performance Assessment Reporting System (CPARS)

Chapter 7 – Performance Assessment Follow-Up

7.1. Follow-up Methods

7.2. Performance Assessment Report (PAR) Follow-up

7.3. Corrective Action Report (CAR) Follow-up

7.4. Contractor Response

7.5. Final COR Recommendation

7.6. Disputes

Chapter 8 - Supporting Guidance and Documentation

8.1. Quality Assurance Evaluation & Standardization (QAV) as Program Management

Office

8.2. Modification Process Flowchart

Chapter 1

Purpose and Scope

1.1. Purpose: This Quality Assurance Surveillance Plan (QASP) has been developed and will be implemented IAW FAR 46.401, DFARS 246.401, PGI 237.172, AFFARS MP5346.103, AFFARS

MP5301.602-2(d), Performance-Based Services Acquisition (PBSA) guidelines and AFI 63-138. It is designed to provide an effective surveillance method of monitoring contractor performance for each listed objective on the Service Summary (SS) in the contract as well as all contract requirements. This QASP provides a systematic method for observing and documenting the services that the service provider is required to furnish and not the details of how the service provider accomplishes the work.

1.2. Scope: This plan covers the Base Operating Support (BOS) functional areas for the Vance

AFB BOS contract.

1.2.1. This QASP is based on the premise that the service provider, not the Government, is responsible for performance management and quality control actions to meet the terms of the contract.

Chapter 2

Performance Assessment Planning & Preparation

2.1. Planning and Preparation: The planning and preparation phase of contractor performance management should be a standard process conducted by the Functional Commander/Director

(FC/FDs), Chief Contracting Officer Representatives (CORs) and CORs over the course of the contract execution. There are several tools and resources available to the Government to aid in the management of the contractor’s performance. The first stop in the planning and preparation phase for the FC/FDs, Chief CORs and CORs should be the Performance Work Statement (PWS). This document will provide the guidelines of what services have been contracted by the Government and more importantly what levels of service are expected and acceptable. The guidelines by which to manage the execution of these services are outlined within this QASP. In addition to these documents, the FC/FDs, Chief CORs and CORs should familiarize themselves with the contractor’s policies and procedures in order to obtain a solid understanding of how the contractor manages their people and programs. The best way to obtain this information is through regular partnering meetings with the contractor.

2.1.1. Partnering: Partnering is the creation of a government/contractor relationship that promotes achievement of mutually beneficial goals during a government contract period or project.

Partnering involves people who believe cooperation is the most effective and efficient way to achieve their goals. The ultimate goal of partnering is to change the “us vs. them” attitude and create the “we” relationship. Nevertheless, partnering shall always operate within the framework of the “arms-length relationship” prescribed in the Guide for the Government-Contractor

Relationship.

2.1.2. The most important step in partnering is for each party to first seek to understand the goals, objectives, and interests of the other party. This will create a “win-win” situation. Efficient partnering has specific attributes that contribute equally to its success. They are common goals, communication, problem solving, and synergies. These attributes seek to accomplish the ultimate goal of cooperation and shared vision.

2.1.3. The Defense Acquisition University (DAU) website offers a Continuous Learning Course

(CLC045) called Partnering. It is highly recommended that each FC/FD and COR at least browse through this course for a better understanding of the concept of partnering and its effects on working relationships.

2.2. Roles and Responsibilities:

2.2.1. Quality Assurance Evaluation and Standardization (QAV). QAV serves as the Program

Management Office responsible for enforcing all policies regarding management, execution, and oversight of the contract as delegated by 71 FTW/CC. Additionally, 71 FTW/QAV is responsible for ensuring that contract surveillance, evaluation and reporting practices are standardized and enforceable across all areas of the Vance BOS Contract. 71 FTW/QAV serves as the action office for all contract compliance and service issues that cannot be resolved at the FC/FD or Group level.

2.2.2. Program Manager. The Chief of QAV has been designated as the Government Program

Manager (PM) for the Vance BOS Contract by the 71 FTW/CC. The PM is responsible for management oversight of all contracted activities. Specifically, the PM shall be responsible for monitoring cost, schedule, technical performance, and risk assessment/mitigation on the contract as delegated by 71 FTW/CC. Develops, updates, implements, and executes plans and programs to provide for the standardization and evaluation of 71 FTW Quality Assurance Program and contract compliance efforts.

2.2.2.1. Contractor Interface: The PM and CO shall serve as the only points of contact between the

Contractor PM and the Government. Any issues requiring wing leadership involvement shall first be presented to the Government Program Manager for disposition. However, contract interpretation can only be done by a warranted contracting officer.

2.2.2.2. Technical Performance: The PM shall oversee technical performance of the contract IAW the PWS. The PM, with the Administrative Contracting Officer (ACO), shall serve as an advisor to the FC/FD with respect to acceptable levels of performance by the contractor. Unacceptable levels of performance by the contractor shall be handled IAW this QASP.

2.2.2.3. Financial Management: The PM shall be responsible for assessing the contractor;’sfinancial execution of the contract and reporting this information to wing leadership on an as needed basis.

2.2.2.4. Risk Assessment: The Program Manager shall work with the FC/FDs to assess overall program risks for SS items not being met. Once a risk assessment has been made, the PM shall work with the FC/FD and ACO to establish an appropriate risk mitigation plan to alleviate the non-conformance. NOTE: The FC/FD retains all responsibility for the success or failure of the contracted function, the same as if the contracted function was a governmental activity. Issues regarding contract requirements not being met should be addressed between the Chief COR, FC/FD and the Contractor’s director overseeing the activity in question prior to escalation to the

PM. However, when circumstances arise that cannot be worked at the FC/FD level, the FC/FD should notify the PM and ACO for determination/assistance.

2.2.3. Chief of Contracting Office (COCO). Ensures a Quality Assurance Program Coordinator

(QAPC) is appointed to integrate the quality contract requirements into the quality assurance program. The COCO shall ensure functional area leadership receives tailored training for PBSA and Quality Assurance (QA). The COCO shall ensure QAPC individual training plans are tailored to the local mission and local QAPC responsibilities. The COCO shall ensure consistent guidance to customers is given by the QAPC and other functional areas such as the Acquisition Centers of Excellence or similar offices. The COCO will serve as the alternate for the QAV Program Manager in the event of extended absence if no other alternate or deputy has been appointed.

2.2.4. Administrative Contracting Officer (ACO). Appointed (warranted) government agents authorized administer contracts. The ACO is the only person authorized to contractually obligate the government or direct the contractor. The ACO delegates quality assurance responsibilities to

CORs, giving them the authority to formally evaluate and accept contractor services.

2.2.4.1. Contract Interpretation: The CO is responsible for contract interpretation and shall serve as the advisor for developing incentives/remedies as appropriate tied to performance objectives and performance thresholds set forth in the contract. The CO shall have final decision authority regarding contract language disputes with the Contractor.

2.2.4.2. Contract Administration: The ACO shall be responsible for overall contract administration overseeing contract modifications and contract file documentation. The ACO shall ensure that all contract documentation is maintained IAW applicable governing directives and AFIs, and copies of Corrective Action Request are provided to the Program Manager.

2.2.4.3. Quality Assurance: The ACO is responsible for ensuring appropriate quality assurance has been conducted before accepting services under the contract.

2.2.4.4. Training: The ACO shall be responsible for ensuring functional area leadership receives tailored training for PBSA and Quality Assurance. The ACO is responsible for the content of all contract-specific training, including Phase II training.

2.2.5. Functional Commander/Functional Director (FC/FD). The FC/FD is the government’s functional authority for the contracted function. The FC/FD retains all responsibility for the success or failure of the contracted function, the same as if the contracted function was a governmental activity. The FC/FD is responsible for assessing the contractor’s performance and managing the requirements over the life of the contract. Specifically, the FC/FD shall:

a) Shall attend training within 30 days after commencement of duties. This training is scheduled by the QAPC and will be tailored to the FC/FDs needs. The training may be delivered by the QAPC, ACO or COCO, with QAV participation.

Review all pertinent documents to include: the PWS, the contractors, proposal, the QASP, service provider quality documents, their training plan, required regulations/publications, service provider and government strike plan annually, and their areas CPAR report.

b) Communicate expectations to the service provider’s Program Manager or area director as applicable.

c) Work with the ACO, through 71 FTW/QAV, in developing or modifying procedures in order to meet governmental expectations and contract standards.

d) At a minimum, when serving as the COR supervisor the FC/FD must perform the COR supervisor duties recapped in QASP Para. 2.2.6.

e) Identify mission essential services, to include developing the necessary documents in accordance with DFARS 237.76, Continuation of Essential Contractor Services .

2.2.6. COR Supervisor.

2.2.6.1. Must have a WAWF account and be a registered DoD JAM/SPM user.

2.2.6.2. Must provide a qualified COR nominee and must review and approve COR nominations in JAM/SPM.

2.2.6.3. Must conduct regular review of COR input into JAM/SPM and follow up as needed.

2.2.6.4. Ensure performance of COR duties and participation in the pre-award process are addressed in annual performance evaluations.

2.2.6.5. Ensure the COR completes the OGE 450 if required. Ensures there is no personal conflict of interest with performance of COR duties.

2.2.6.6. Ensures adequate resources are provided for performance of COR duties prior to award and throughout contract performance.

2.2.6.7. Ensures the COR attends all required initial and refresher COR training.

2.2.7. Quality Assurance Program Coordinator (QAPC) normally from the contracting activity, selected to coordinate the Quality Assurance program. The QAPC is responsible for training the functional commanders and CORs on surveillance methods and overseeing the effectiveness of the quality assurance surveillance program. The QAPC works with CORs, the ACO, and FC/FDs to identify, document, and troubleshoot contract compliance issues. Standardizes and evaluates

Contractor Performance Assessment Reporting System (CPARS) inputs from COR’s and FC/FDs.

2.2.8. Chief Contracting Office Representative (COR). Responsible for ensuring surveillance and documentation of contract performance and report noncompliance or abnormalities to the

Functional Commander(s), the Wing Chief COR, and the ACO. Specifically, the Chief COR shall:

a) Verify that the service provider meets contract obligations as specified in the contract.

b) Review/document QASP, functional area appendices and checklists as required for adequacy. Responsible for the technical adequacy, accuracy and sufficiency of functional area QASP Appendices describing specific technical approach and methodology for surveillance. Forwards updated QASP appendices and checklists to QAV and QAPC as changes occur.

c) Review the service provider’s quality control program for acceptable quality level and recommends acceptability to the ACO through the Functional Commanders.

d) Ensure each COR is initially evaluated to determine qualifications, experience, and ability to accomplish scheduled inspections and contract surveillance functions. It is the responsibility of the Chief COR, and coordinated with QAPC, to ensure all required training, such as DAU Course COR 222 (COR Training) and CLM 003 (Ethics), and

Phase I & II, is completed by all new CORs. Also ensures refresher training for these courses is attended annually. Chief COR must attend AETC Chief COR Training (Block

3) either at AETC or locally. All required training shall be scheduled and coordinated through the QAPC.

e) Ensure that each COR is performing their duties IAW this QASP, MP5301.602-2(d) and the ACO issued COR Designation memorandum.

f) Assist the ACO in inspecting and surveilling PWS 2 requirements of the contract managing the applicable government-furnished property clause of the contract.

g) Ensure a monthly summary of all COR surveillance activities is reviewed and signed by the Functional Commander(s) and is provided through the QAV office, to the ACO, no later than the 5 th workday of the month. The Chief COR shall mark whether performance was satisfactory or unsatisfactory for the month.

h) Review of contractor proposals. All facets of the proposal, within the technical expertise of CORs, will be evaluated to include number of personnel, skill level of personnel, man-hours proposed, and all associated costs.

i) Assist HQ AETC/A4PM, Functional Commander(s), QAV, and/or the ACO in determining contract cost estimates, if requested.

j) Perform surveillance activities as required.

k) Review and evaluate service provider-submitted value engineering change proposals.

l) Provide assistance to the Wing Safety Office, or equivalent, in mishap and incident reporting.

m) Review service provider regulations and/or instructions prior to acceptance and publication.

n) Verify the statistical information provided by the service provider that concerns the standards specified in the Performance Work Statement (PWS) or appendix of the contract.

o) Gathers past performance information and provides to ACO through QAV.

p) Attend monthly QAV/Chief COR meetings. If unable to attend, an alternate shall be assigned.

q) Review service provider and governmental strike plan annually. Submit updated/revised governmental strike plan as required to the ACO.

r) Ensure COR QA files are maintained as required.

s) Ensure the COR(s) use the DoD Wide Area Workflow (WAWF) Joint Appointment

Module (JAM) Surveillance and Performance Monitoring (SPM) at http://wawf.eb.mil/

2.2.9. Contracting Office Representative (COR). The CORs primary purpose is to surveil and document. CORs are the “eyes and ears” of the Wing Commander, Functional Commander(s), Program Manager, and ACO relative to the actual application of the contract; however, they are not a quality control function. Another important part of the CORs responsibilities is to “partner” with the contractor regularly to ensure open communications between the Government and contractor. The COR is to be objective, fair, and consistent in evaluating service provider’s performance, against scheduled and regulatory requirements and the terms of the contract. CORs will not direct work or re-accomplishment of work, change the contract, or formally interpret the contract. The ACO resolves these types of issues. CORs are prohibited from training or advising the contractor and/or performing contracted tasks or quality control work.

However, other government employees may do this under certain circumstances. If requested by the contractor, the CO, under the partnering principle, may direct other government employees to train or advise the contractor to help them get to a level of competence required by contract standards or in support of new equipment the government has provided them. Specifically, the

COR shall:

a) Know the specifications and requirements of the contract.

b) Know and maintain proficiency in contract surveillance procedures and requirements.

CORs will maintain technical competency in their functional areas, maintain proficiency in contract surveillance techniques, and assist with formulating the PWS and observation area inspection guides. CORs will review/approve contractor-developed instructions noting necessary changes/updates in Quality Management Information System (QMIS) prior to signature and implementation as directed by the Chief COR. QMIS is referred to as the “SMART” system in PWS Section 2. CORs will accomplish contract surveillance by evaluating and documenting service provider performance and inform the Chief COR and FC /FD when service provider performance does not meet performance thresholds.

c) Know and apply the procedures for documenting surveillance.

d) Perform surveillance according to their surveillance schedules.

e) Maintain scheduled competency in his/her assigned surveillance areas.

http://wawf.eb.mil/

f) Attain qualification in the appropriate areas before performing evaluations, inspections, or surveillance duties unsupervised.

g) Review the deficiency and mishap service provider reports for accuracy, adverse trends, and mission accomplishment. Additionally, reviews reports to higher headquarters for possible indicators of performance trends.

h) Serve as a member of the source selection team when selected.

i) Develop surveillance schedules.

j) Review service provider regulations and/or instructions prior to acceptance and publication.

k) Be familiar with emergency procedures to be implemented if default or strike interrupts service provider performance.

l) Ensure service provider does not answer higher headquarter suspense’s and requests without prior approval from local leadership.

m) Use proper governmental and contractor chain of command when elevating issues to upper management.

n) Maintain a QA file with all COR training documents/certificates, Contract documentation

(such as copy of the Contract, Contract modifications, PWS, QASP, etc…), all surveillance documentation, and any other documents pertinent to this contract.

o) Process COR’s self-nomination and maintained COR training documents/certificates in the JAM/SPM Modules (https://wawf.eb.mil/ ).

p) Ensure they have received full COR training and COR designation memorandum, with the authorized roles and responsibilities signed by the ACO, prior to performing surveillance duties.

2.2.10. Performance Management Council (PMC). The purpose of the Performance Management Council is for the Government and the service provider to collectively review the service provider’s performance on a quarterly basis. Areas addressed are: SS performance, surveillance results, contract administration and current issues. The areas addressed may change depending on current leadership concerns/issues. The slides for this meeting are created by the Chief COR and are partnered with the FC and their contractor counterpart prior to submission to QAV. Slides are submitted to QAV quarterly.

2.2.10.1. The meetings are scheduled on the wing’s master meeting schedule and therefore all responsible parties are aware of their frequency. QAV may send courtesy meeting reminders to council members.

2.2.10.2. Members of the PMC include, but are not limited to: the service provider, the Wing CC, Group CCs, QAV, the ACO, FC/FDs, and Chief COR.

https://wawf.eb.mil/

2.2.11. Multi-Functional Team (MFT) Roles and Responsibilities. The MFT is a customer-focused team instituted under the authority of Senior Leadership. The purpose of the MFT is to create an environment that shapes and executes an acquisition. The emphasis is on teamwork, trust, common sense, and agility. The goal is to obtain efficiencies, improved performance, and cost saving throughout the acquisition life cycle. The MFT is composed of stakeholders in the acquisition. These stakeholders are responsible for this acquisition throughout the life of the requirement. See MFT Member Appointment letter for MFT role and responsibilities in contract file and as stated within the QASP, which is IAW AFI 63-138, Chapter 2 and DoDI 5000.02.

2.3. Required Training. IAW MP5301.602-2(d), all COR nominees must complete COR training which is done in two (2) parts. COR Part 1 Training consist of mandatory Defense

Acquisition University (DAU) on-line courses as specified in OUSD (AT&L) Memorandum, 29

Mar 10 and the type requirement (Type A, B or C) as determined by the ACO, COR Supplemental training as determined by the ACO and QAPC to include the Initial Contract Surveillance Phase I training (when applicable); and COR Part 2 Training also known as Phase II training, which consist of Contract Specific training provided by the ACO or ACO’s designee. Based on the ACO’s determination, all CORs will complete the following DAU training courses, and COR

Supplemental training:

1. DAU COR 222, Contracting Officer’s Representative Course

2. DAU COR 206, COR in a Contingency Environment (when applicable)

3. CLM 003, Ethics Training for Acquisition Technology and Logistics (Or approved Air

Force Equivalent)

4. Joint Ethic Regulation DoD 5500.7-R (review/understanding of regulation)

5. Additional/Supplemental Training as required by CO and QAPC

6. Combat Trafficking in Persons at http://ctip.defense.gov .

7. Seven Steps to the Services Acquisition Process at http://sam.dau.mil/ .

8. OPSEC 107

http://www.acq.osd.mil/dpap/policy/policyvault/USA005569-09-DPAP.pdf http://www.acq.osd.mil/dpap/policy/policyvault/USA005569-09-DPAP.pdf http://ctip.defense.gov/ http://sam.dau.mil/

2.3.1. Combating Trafficking in Persons. DoD has a zero tolerance policy for human trafficking.

CORs are the first line of defense in the battle against human trafficking and must complete

Combating Trafficking in Persons training annually. As the COR monitors the contractor and its employees, Combating Trafficking in Persons should rank among the COR’s chief priorities.

CORs must be diligent in ensuring that contractors and contractor personnel are not trafficking in persons. During surveillance, the COR will monitor the contractor’s performance regarding trafficking in persons for compliance with all that is required IAW FAR clause 52.222-50, Combating Trafficking in Persons. The COR must inform the Contracting Officer if the contractor, contractor personnel, subcontractor, or subcontractor personnel have failed to comply with the requirements of the clause at FAR 52.222-50. After receiving this information, the Contracting

Officer shall provide information for any investigation and enforcement to:

Program Manager

DoD CTIP Law

Enforcement and Support

OUSD (P&R) DHRA

4800 Mark Center Dr Suite 06J25-01

Alexandria, VA 22350-4000

Reports may also be made:

On-line at http://ctip.defense.gov/;

On-line via the National CTIP Hotline website at http://www.polarisproject.org/;

By e-mail to CTIPReports@OSD.Pentagon.mil; or

By phone to the National CTIP Hotline at 1-888-373-7888.

Note: The COR must report any suspected violations or activities to the Contracting Officer (See

FAR Subpart 22.17 and DFARS Subpart 222.17).

Note: CORs should not personally investigate suspected incidents of Trafficking in Persons but should forward all reported or suspected violations to the Contracting Officer immediately.

Note: The Contractor must take appropriate action should a violation occur. For corrective procedures and remedies for non-compliance, see FAR 52.222-50.

http://ctip.defense.gov/%3B http://www.polarisproject.org/%3B

Chapter 3

Performance Assessment

3.1. Performance Assessment. Performance assessment is evaluating, assessing, and documenting the contractor’s performance IAW the overall QASP. The premise behind performance assessment is that the contractor, not the government, is responsible for managing and ensuring that performance meets the terms of the contract. It is the government’s responsibility to monitor performance and hold the contractor accountable for performance. All areas of the contract shall be surveilled and the performance of the contractor shall be assessed and reported in accordance with this QASP.

3.1.1. Service Summary (SS). The SS identifies the major service requirements the USG is acquiring and provides a basis for measurement between actual SP performance and specific contract requirements. The COR in partnership with the SP, will verify adequacy of performance for the critical or major requirements using any of the quality assurance methods delineated in

QASP paragraph 3.2. The USG has the right to inspect all services, not just Performance Objective

(PO) items. The following table shows the Performance Thresholds and frequency of surveillance for PWS Section 2.

ITEM PERFORMANCE

OBJECTIVE

REFERENCE

PARAGRAPHS

PERFORMANCE THRESHOLD

A Ensure positions for each service area are filled.

All PWS

Sections

Maintain an overall fill rate of 90%

B Ensure timely fill of key personnel positions with a qualified employee

All PWS

Sections

No more than 60 days to fill a key personnel position with a qualified employee

C Provide good customer service and maintain high customer satisfaction rates.

All PWS

Sections

Customer satisfaction will be maintained at a minimum level of 4 on a scale of 1 to 5 (l lowest – 5 highest) on a monthly basis as determined by validated customer comment cards, customer surveys, and other feedback mechanisms. In addition, no more than 3 validated customer complaints

(within each respective PWS section) received per month.

D Ensure suspenses are met within the prescribed timeframes.

All PWS

Sections

No more than 5% of all suspenses (within each respective PWS section) will be met with no greater than an 8 hour delay.

3.2. Surveillance Methods. The COR will ensure contract compliance through various surveillance methods. These methods may include but are not limited to: periodic inspections, random sampling, customer feedbacks/complaints, service provider metrics, 100% surveillance and quality indices third party audits.

3.2.1. Periodic Inspections: These items are inspected using periodic surveillance (daily, weekly, monthly, quarterly, etc.) as determined by the COR. The results of the periodic surveillance inspections may be used as the basis for actions (other than payment deductions) towards the contractor. In such cases, the Inspection of Services clause becomes the basis for the contracting officer’s actions. Periodic inspections may be conducted via over the shoulder inspections, spot sampling, or review of the contractor reports and other deliverable documents.

3.2.2. Random Sampling: Random sampling is an appropriate method for frequently recurring tasks. A typical use of this surveillance method is lot size sampling. Initially, the COR will evaluate randomly selected samples of the activity to determine the acceptability of the entire requirement.

This method reduces the time the COR must spend on surveillance duties, and yet still gives a reasonably accurate picture of a service’s overall acceptability.

3.2.3. Customer Feedbacks/Complaints: Customer inputs will be registered in QMIS allowing

FC/FDs to review as required. The Chief COR, or designated COR, should review all customer feedbacks.

3.2.3.1. When a Customer Feedback SS is present, the COR must collect and review the feedbacks prior to the service provider unless the feedbacks are directly entered into QMIS by the customer.

This may be accomplished by placing a lock box in each facility that offers hard-copy customer feedbacks. The COR will maintain the key to this lock box. As a courtesy, before removing the customer feedbacks, the COR will notify the service provider that he/she is removing them. After COR review, the feedbacks will go to the service provider for review/action. The COR should review and return the feedbacks to the service provider within 24 hours of receipt. If the service provider has a contractual requirement to answer negative feedbacks within a certain amount of time, that time will not begin until the service provider receives the feedback from the COR.

3.2.3.2. If there is a requirement for negative feedback follow-up, the COR must monitor that follow- up. The COR will validate the actual occurrence and/or the results if the situation requires further investigation. The Chief COR will consider the actions taken by the service provider and will determine if the response is expected to resolve the customer’s complaint. If the action fails to address the problem or is inadequate, the Chief COR and FC/FD shall contact the ACO through

QAV for further action.

3.2.3.3. If a SS requiring a performance threshold on customer feedbacks is in the PWS and minimal feedbacks are received, on a monthly basis, each COR will contact government customers for their feedback on the service providers’ performance. The number of feedbacks received will be determined by the need in each department.

3.2.3.4. Customer Complaints Resolution: Actions taken in support of a specific SP or Government concern or problem that impacts operations. Usually results in Government-to-SP or SP-to-SP process deficiency resolution (feedback). NOTE: Customer complaints, once validated, are a means of surveillance and are alternative to formal surveillance methods. The COR is the point of contact and must collect all customer complaints. The AF Form 714, Customer Complaint Record, or a similar locally devised form may be used for this purpose. All complaints are validated and any resulting resolution of such complaints must be documented with the information required on the customer complaint form. Customer complaint forms become a permanent part of the COR surveillance records. The COR will contact each customer involved with this contract, periodically to assure there is an understanding of the contract requirement by all appropriate personnel and they have a sufficient number of complaint forms.

3.2.4. Service Provided Metrics: Each department or functional area has SSs which are part of the

PWS. They are the major areas of concern for each functional area. However, these are not the only requirements that are measured. Every “shall” statement within the PWS should be measured.

This includes Section 2 of the PWS which is applicable to all areas. SSs are found in a table at the end of each PWS section. They are broken down by Performance Objective, PWS Reference, and

Performance Thresholds. This table details what and how often each requirement is inspected or surveilled. Changes to SSs can only occur with a modification to the contract.

3.2.5. One-Hundred Percent Inspection: When this type of surveillance is used, the COR must inspect and evaluate the contractor’s performance each time it is performed. The result of the contractor’s overall performance is then evaluated to determine acceptability of the lot.

3.2.6. Third Party Audits or Assessments: Third party audits or assessments refer to contractor evaluation by a third party organization that is independent of the Government and the contractor.

All documentation supplied to, and produced by, the third party should be made available to the government by the contractor.

3.3. Other Surveillance Methods. The following are types of surveillances that may be used throughout the contract period to evaluate service provider performance when structured surveillances do not exist.

3.3.1. Budget Monitoring: The Chief COR will monitor service provider’s compliance and advise him/her of budgetary trends when needed. CORs will surveil supply listings to ensure supplies ordered are those needed to perform the mission and that funds are provided for these purchases. The COR will ensure the service provider includes the COR on routing for all purchase requests. The COR will ensure service provider includes governing reference for authorization of purchases on all purchase requests. Financial plans and unfunded requirements developed by the service provider will be surveilled by the COR. The

COR will be included on these routings as well as all purchase requests.

3.3.2. Exercise/Contingency Surveillance. The service provider’s participation in exercises/ contingencies will be surveilled by a COR assigned to the Wing Exercise Evaluation Team

(WEET). CORs may be appointed to the WEET. As a minimum, service area related exercises will be surveilled using appropriate checklists.

3.3.3. PWS Section 2 Surveillance. FC/FD and Chief COR will determine the frequency of these inspections. Inspections of Section 2 will be documented in QMIS just as all other inspections are documented. These inspections can be conducted with other inspections but should include verbiage of what specific requirements from Section 2 were covered during the inspection.

3.4. Surveillance Schedule. The Chief COR will develop a schedule of surveillance based on the requirements of FAR Part 46, MP5346.103 and MP5301.602, and this QASP. This schedule shall include surveillance of contractor performance as appropriate, outside of the CORs normal duty hours at least monthly (i.e., all shifts as required). Surveillance schedules should specify all work requiring surveillance and should specify the method of surveillance. They should be prepared in conjunction with the preparation of the statement of work. The surveillance schedule will be completed no later than seven calendar days prior to the beginning of the period it covers, and a copy will be sent to the ACO and QAV. The FC should sign the schedule prior to submitting.

The schedule is “FOR OFFICIAL USE ONLY” and is not releasable to anyone other than authorized government personnel.

3.4.1. Surveillance Schedule Changes. The Chief COR will forward changes to monthly surveillance schedules to the ACO and the Wing Chief COR, as changes occur.

3.4.2. Determining surveillance frequency. During performance assessment planning, the Chief

COR shall consider using operational risk, service complexity and criticality as factors in deciding the performance assessment plan from month to month. If a particular function of the contractor’s performance has a continuing record of acceptable performance, and unacceptable performance would not likely result in loss of life to AF personnel or damage to government property, surveillance of that function may be reduced. If contractor performance of a function is less than satisfactory, surveillance of that function should be increased. When this is determined to be appropriate, the Chief COR, with FC/FD and PCO/ACO approval, will adjust the surveillance schedule.

3.4.3. Monthly Surveillance Shortfall. If minimum monthly surveillance requirements cannot be accomplished due to equipment non-availability or special circumstances, the reason(s) for missing inspection will be documented on the end of the month’s summary report.

3.4.4. Waiver of scheduled observation area inspections. If the Chief COR determines a contractor work center has a continuing record of acceptable performance, its scheduled observation area inspection may be waived for one observation period (month, quarter, etc.), if approved by the

FC/FD and the ACO. The Chief COR will include comments concerning the waived observation area inspection in the monthly summary. Surveillance will never be waived for more than one prescribed period and the surveillance frequency will go back to normal if contractor performance shows signs of less than acceptable performance.

3.4.5. The Chief COR will monitor surveillance results to ensure no particular work areas/sites and/or office areas are being disproportionately inspected. The Chief COR may, however, deviate from random selection to ensure a broad cross-section of the service area is being inspected to investigate potential problems or if trends indicate a need. It is essential that CORs accomplish sufficient in-depth inspections in all areas to measure the quality of service provider performance and provide an effective measurement/assessment to ensure overall performance meets contract requirements. Compliance with applicable directives prescribed by the PWS, service provider-developed Quality Control Program (QCP), Contract Sections, and plans/instructions will be part of the surveillance.

3.5. Surveillance Ratings: A discrepancy is any validated inspection finding or observation.

Each BOS FC/FD and Chief COR will determine then partner with the service provider on what constitutes a major versus a minor discrepancy for their areas. Many of the SS items in our PWS describe the type of discrepancies that constitute a failure to comply with contractual requirements. Any discrepancy already defined in the PWS as a failure is considered a major discrepancy. Additional major discrepancies will be described by the FC/FD and Chief COR, but are primarily defined as any contractor actions that could result in a violation of safety, security, protection of resources, or law.

3.5.1. Major discrepancies include but are not limited to: A step serious enough to adversely affect the performance of the equipment involved is omitted or improperly completed; violation of federal, state or local laws; DoD and Air Force environmental protection policies and directives;

T.O. violations, to include not using required TOs to perform tasks, violations of mandatory directives and supplements when such may cause damage to government property or injury to government personnel; improper use of tools or use of outdated data when such may cause damage to government property or injury to government personnel; or accomplishing procedures without training or qualification on the task.

3.5.2. A minor discrepancy is any failure to comply with the contract that is not already identified as major, does not compromise Vance’s mission, and is easily correctable.

3.5.3. Baselines for number of minors that constitute a major are determined by the FC/FD and

Chief COR. Minor discrepancies that consist of a grouping of like deficiencies may be documented as one or several discrepancies. When grouping deficiencies, CORs will ensure surveillance techniques are applied consistently across inspection areas. Surveillance inspections will be rated as either acceptable or unacceptable.

3.5.4. "As Observed" inspections are unscheduled inspections. They occur when discrepancies or deficiencies are observed or discovered that are not directed and reported. “As Observed” inspections are rated as conform and non-conform. “As Observed” inspections should be documented as a Performance Assessment Report (PAR) or Corrective Action Report (CAR) as deemed appropriate by the FC/FD or COR. These findings however, will not count against the contractor’s findings rate. If “As Observed” findings highlight a potential performance issue, consideration should be given to adding the task or tasks to the COR’s regularly scheduled inspections. Once included in the regular inspections, any further discrepancies shall be noted and will count against the contractor’s finding’s rate.

3.6. Assessment of Contractor Quality Control Programs: All areas of the contract discuss the contractor’s quality control but PWS Section 2 is most specific. It is critical that each COR become knowledgeable with Section 2 because it includes all the miscellaneous “shall” statements that the service provider must accomplish.

3.6.1. Each COR will be knowledgeable on the contractor’s Quality Control Plan (QCP). The contractors QCP for Vance AFB is located in QMIS under Quality Documents (QDs). Each department within the contract has individual Quality Documents. The COR should ensure they are included as a “reviewer” on the routing of any new QDs and QD changes.

3.6.2. The CORs’ primary job is to observe and document the contractor’s performance. However, it is very easy for a COR organization to become the contracto’rs quality control without realizing

it. The COR organization must continually evaluate their role and ensure the contractor is truly responsible for the quality of services provided. Some warning signs that a COR organization has become, or is becoming the contractor’s QC include: CORs finding problems that the contractor’s

QC isn’t, contractor management only fixes problems that are identified by the CORs, contractor employees are more intimidated by CORs than the contractor’s QC, and the contractor is not proactive on quality issues.

3.6.3. Inspections are entered into QMIS for COR scheduled inspections of service provider

Quality Controls. A negative PAR will be documented when findings are observed using either a

Section 2 or contractor QD reference. QC over-the-shoulder or QC after-the-fact inspections will be accomplished with other scheduled inspections by each COR. The FC/FD and COR organization will also ensure “Partnering” with the service provider and their QC personnel is accomplished. The frequency is determined by the FC/FD and Chief COR. Partnering is critical to understanding and communication between the Government and service provider. It is highly recommended that the FC/FD attend each partnering meeting.

3.7. Assessment of Non-Service Summary (SS) Items. All other tasks required under the contract may be inspected on a random basis at a frequency determined by the FC/FD, ACO, and/or

COR per the Inspection of Services Clause in the contract.

3.8. Assessment of Service Provider Training Programs. The USG has the right to inspect all tasks within this PWS not just the Performance Objectives (POs) listed in the Service Summary.

All other tasks required under the PWS may be inspected on a random or periodic basis at a frequency determined by the Chief COR, ACO, and/or COR per the Inspection of Services Clause in the contract. The COR shall perform Scheduled Inspections to effectively monitor compliance with the service provider’s training plan to ensure that training provided meets applicable qualification and training standards as well as remains current.

3.9. Remedies for Unacceptable Performance. In accordance with the contract inspection of services clause, if any of the services do not conform to contract requirements, the Government may require the contractor to perform the services again in conformity with the contract requirements, at no increase in contract amount. This includes, but is not limited to, termination of contractor personnel and recruitment of substitute personnel that are equally qualified within established time frames. When the defects in services cannot be corrected by re-performance, the

Government may:

a) Require the contractor to take necessary action to ensure that future performance conforms to contract requirements.

b) Reduce the contract line item price to reflect the reduced value of the services performed.

c) If the contractor fails to promptly perform the services again or to take the necessary action to ensure future performance in conformity with contract requirements, the Government may:

i. By contract or otherwise, perform the services and charge to the contractor any cost incurred by the Government that is directly related to the performance of such service;

or

ii. Terminate the contract

NOTE: All remedies listed above can only be levied by the CO.

Chapter 4

Performance Assessment Analysis

4.1. Data Review: Chief CORs shall review performance data with COR inputs and coordinate with the FC/FD on recommendations prior to formalizing a corrective action plan. Chief CORs should use trend analysis to accurately assess the contractor’s performance within each respective area.

4.2. Trend Analysis: Statistical data for COR inspection rates are maintained in QMIS and monitored by each COR. Chief CORs should use this historical data for trend analysis in order to accurately identify negative and positive trends in contractor performance. Trend analysis should be used by the Chief COR when making recommendations to increase or decrease surveillances in an individual area and when making determination on closing PARs and CARs. This method of surveillance should be used regularly and continually by COR personnel to monitor the contractor’s on-going performance over time. Data for tracking trends can be gathered from all other evaluation sources and methods as well.

4.2.1. Negative Trends. If negative trends in contractor performance are noted through trend analysis, the Chief COR shall inform the FC/FD and consideration should be given to increasing surveillances in these areas until the trend has reversed.

4.2.2. Positive Trends. If positive trends in contractor performance are noted, consideration may be given to reducing the number of surveillances in that specific area. Positive trends in findings rates are good indicators that contractors QC programs are healthy or that contractor corrective actions are working.

Chapter 5

Documentation of Service Provider Performance

5.1. Documenting Performance: Service provider performance shall be thoroughly documented by the CORs. All PARs shall be input into QMIS by the COR for the contractors review and response. PARs should also be used to document exceptional performance by the contractor as opposed to just discrepancies.

5.2. Discrepancy Documentation: All discrepancies documented during COR surveillances/ inspections shall be documented in QMIS. Due to the automatic notifications in QMIS, the QMIS submission shall serve as the service provider acknowledgement of the PAR. However, as a courtesy, the COR should attempt to contact the service provider prior to entering the discrepancy into QMIS to inform them of the forthcoming PAR. A copy of the PAR shall be forwarded to the ACO, through the Wing Chief COR, with the monthly PAR report. These originals will be maintained by the

ACO’s office and will be considered a permanent part of the contract.

5.2.1. Required Information: Specific PWS and/or AFI paragraph references shall be used when documenting non-conformance as applicable.

5.3. Repeat Write Ups: CORs shall review completed PARs to identify repeat discrepancies.

When repeat discrepancies are found, the COR issues another PAR clearly annotating the deficiency as a repeat. The FC/FD and Chief COR will determine how many repeat findings constitute a CAR. The FC/FD and Chief COR will determine, based on the functional area, what constitutes a repeat.

Chapter 6

Performance Assessment Reporting

6.1. Scheduled Inspections and Observation Reports: Scheduled Inspections and Observations are entered into QMIS and on the monthly tracking sheet, PAR Summary Report, as a method of checks and balances for completeness of monthly inspection requirements. These reports are sent to QAV, then the ACO on the 5th business day of the month.

6.2. Additional Reporting Requirements: Additional required monthly reports include PMC

Quad Charts. These reports are due to QAV by the 10th business day of each month. They are compiled, as a whole, by QAV, and sent to Wing Leadership, the ACO, (and AETC/A4PM (for

MAQ only)) by the 15th business day of the month. Depending on wing leadership requirements, changes to the Quad Chart and/or additional data/reports may be requested by QAV or the ACO.

6.3. Contractor Performance Assessment Reporting System (CPARS). During the source selection process, every contractor’s past performance on other contracts is critical selection criterion. Due to the competitive nature of the source selection process, contractors know that a proven track record is a big key to securing future contracts and they work hard to preserve their credibility and performance record. The CO, QAV, FC/FD, and Chief COR are responsible for completing the Contractor Performance Assessment Report (CPAR) annually. The most common tool used to enter this data is the CPARS. This system receives reports and tracks contractor performance. If there are significant changes in contractor performance, the assessing official

(FC/FD and CO) can do an out-of-cycle CPAR.

Chapter 7

Performance Assessment Follow-Up

7.1. Follow-up Methods: Follow-ups are accomplished using several methods that include:

follow-up inspections accomplished behind the service provider’s QC, follow-up inspections on individual training records, routine/daily review of service provider responses to COR discrepancies in QMIS, and formal notification of adverse trends in service provider performance and/or failure to meet contract standards.

7.2.