552 ACW E-3 CAT CWD QASP 1 Aug 19.pdf

BY ORDER OF THE DIRECTOR HQ ACC ACQUISITION

MANAGEMENT AND

INTEGRATION CENTER

## Mmmmmm 2019 Revision date – Basic

Quality Assurance Surveillance Plan

552 Air Control Wing E-3 Contract

Aircrew Training and Courseware Development

Solicitation: FA4890-19-R-A020 Contract: FA4890-20-C-####

Pages: 4242

COMPLIANCE WITH THIS DOCUMENT IS MANDATORY

NOTICE: This Document is available in digital format through ACC AMIC/DRQQ

OPR: ACC TRSS/ACQ Reviewed by: ACC TRSS/ACQ

DRUIE D. BARNARD, GS-13, DAF RAYMOND L. CHURCHILL, GS-14, DAF

Program Manager Chief, Training Acquisition Flight

Reviewed by: ACC AMIC/DRQQ Accepted by: ACC AMIC/PKBB

JOHN D. NOBLE, NH-03, DAF TARA M. HOLLOWAY, NH-03, DAF

Quality Assurance Program Coordinator Contracting Officer

Quality Assurance Surveillance Plan 552 ACW E-3 CAT/CWD Solicitation # FA4890-19-R-A020

Contents

1.0 SCOPE

1.1 Authority

1.2 Purpose

1.3 Related Documents

2.0 MANAGEMENT LOCATIONS AND KEY PERSONNEL

2.1 Locations

2.2 Government Multi-Functional Team

2.3 Contractor Key Personnel

3.0 CONTRACT SURVEILLANCE

3.1 Surveillance Methods

3.2 Surveillance Documents

3.3 Deadline to Submit Report

3.4 Addidtional COR Inspections

4.0 CONTRACTOR PERFORMANCE REPORTING

4.1 Non-conformances

4.2 COR Monthly Report

4.3 Annual Contract Performance Assessment Reporting System (CPARS)

4.4 Wide Area Workflow (WAWF)

5.0 COR FOLDER

Attachment 1 – Non-Conformance Report (Example Only) Attachment 2 – Initial Performance Assessment (IPA) (Example Only) Attachment 3 – Sample Monthly Surveillance Report Attachment 4 – Individual Performance Objective Inspection Checklists Attachment 5 – Sample SME Contract Monitoring and Surveillance Report Attachment 6 – Sample SME Nomination Letter Attachment 7 – Sample SME Relief from Duties Letter Attachment 8 – Sample Combination SME Nomination & Relief from Duties Letter Attachment 9 – Guidelines For Courseware Deliverable Evaluation

TABLES

Table 1 – Deliverables Table 2 – Services Summary Table 3 – Non-conformance notifications Table 4 – Monthly Performance Rating Definitions Table 5 – Performance Assessment Report Routine Timelines* Table 6 – COR Folder Tabs & Contents

1.0 SCOPE

The Quality Assurance Surveillance Plan (QASP) is applicable to personnel performing contract surveillance inspections in accordance with Federal Acquisition Regulation (FAR) Part 46, Quality Assurance; Department of Defense Instruction (DODI) 5000.72 DoD Standard for Contracting Officer’s Representative (COR) Certification; DOD FAR Supplement (DFARS) Subpart and PGI 201.602-2 , Contracting Officer Responsibilities; Air Force FAR Supplement (AFFARS) Mandatory Procedures (MP) 5301.602-2(d) Designation, Assignment, and Responsibilities of a COR; Air Force Instruction (AFI) 63-138, Acquistion of Services; and organizational policy. It is designed to provide the Contracting Officer Representative (COR) a systematic surveillance method for each service identified in the Performance Work Statement (PWS). The Program Manager (PM)/Functional Service Manager (FSM) and COR develop the methods for administering and evaluating performance. The Quality Assurance Program Coordinator (QAPC) and Contracting Officer (CO) or Contract Manager (CM), will review these methods, which are then accepted or rejected by the CO.

1.1 Authority

1.1.1 The following FAR provisions and clauses provide authority for this QASP.

1.1.2 Applicable inspection clauses (not all inclusive, see contract for all clauses):

FAR 52.212-4 – Contract Terms and Conditions -- Commercial Items FAR 52.246-4 – Inspection of Services -- Fixed-Price

1.1.3 Applicable FAR Provision information (not all inclusive, see contract for all provisions):

1.1.3.1 FAR 12.301(b)(3) authorizes the Contracting Officer to tailor Clause 52.212-4 in accordance with FAR 12.302 -- Tailoring of Provisions and Clauses for the Acquisition of Commercial Items. In the case of this contract, the PWS is tailored to institute 52.246-4 and require the Contractor to provide a Quality Control Plan as a part of their Management Plan.

1.1.3.2 FAR 46.202-1, Contracts for Commercial Items. This provision places the responsibility for production quality assurance on the contractor. Any in-process audits conducted by the Government must be consistent with commercial practices.

1.1.3.3 FAR 46.202-3, Standard Inspection Requirements. This provision authorizes the use of clause 52.246-4 and require the contractor to provide and maintain an inspection system acceptable to the Government, and make the inspections records available to the Government.

Additionally, this provision gives the Government the right to inspect services while work is in process.

1.1.3.4 FAR 46.202-4, Higher-Level Contract Quality Requirements. This provision, in conjunction with FAR 46.203 and 46.311, authorize the Government to impose the Contractor use a higher-level quality system like ISO 9001 or SAE AS9100. In addition to in-process audits, the COR may audit the Contractor’s use of and the effectiveness of their quality management system.

1.1.3.5 FAR 46.401, Government Contract Quality Assurance. This provision directs that Government contract quality assurance shall be performed as necessary to determine that the supplies or services conform to contract requirements. Quality assurance surveillance plans will be prepared, and should specify the work requiring surveillance and the method of surveillance.

1.1.3.6 FAR 46.407 -- Nonconforming Supplies or Services. This provision specifies the contracting officer rejects supplies or services not conforming in all respects to contract requirements. The contracting officer ordinarily must give the contractor an opportunity to correct or replace nonconforming supplies or services when this can be accomplished within the required delivery schedule; correction or replacement must be without additional cost to the Government.

1.1.3.7 FAR 46.501 – Acceptance (General). Acceptance constitutes acknowledgment that the supplies or services conform with applicable contract quality and quantity requirements, except as provided in this subpart and subject to other terms and conditions of the contract. Acceptance may take place before delivery, at the time of delivery, or after delivery, depending on the provisions of the terms and conditions of the contract. Supplies or services shall ordinarily not be accepted before completion of Government contract quality assurance actions. Acceptance shall ordinarily be evidenced by execution of an acceptance certificate on an inspection or receiving report form or commercial shipping document/packing list.

1.2 Purpose

This QASP provides a systematic method to assess and evaluate the services received from the Contractor, however, this QASP is not considered part of the contract. It defines the policy, procedures, and associated methods used by the Multi-Functional Team (MFT) for planning, preparing, performing, analyzing, and reporting Contractor performance. In the event this QASP conflicts with the contract or task order, the contract or task order shall take precedence.

1.3 Related Documents

1.3.1 552 ACW E-3 CAT/CWD, Contract number FA4890-20-C-####.

1.3.2 Solicition number FA4890-19-R-A020.

2.0 MANAGEMENT LOCATIONS AND KEY PERSONNEL

2.1 Locations

Contract Management and Quality Oversight ACC AMIC/PKB and DRQQ 129 Andrews Blvd (Bldg 664) Joint Base Langley-Eustis VA 23665

Program and COR Management

ACC TRSS/ACQ

130 Andrews St., STE 118 Joint Base Langley-Eustis VA 23665

2.2 Government Multi-Functional Team

2.2.1 Contract Management

2.2.1.1 Contracting Officer (CO). CO duties are detailed in FAR subpart 1.602-2, Contracting Officer Responsibilities, and AFFARS MP5346.103, Contracting Officer Responsibilities. The CO is the only Government agent authorized to award or modify contracts; therefore, the CO is the only person authorized to contractually obligate the Government. The CO is also the individual who will sign any notifications to Contractors regarding performance issues and accepts Government surveillance methods used under the award. The CO approves/appoints all CORs assigned by COR Management.

NOTE: The CO can be reached at DSN: 574-3222 or Commercial 757-764-3222.

2.2.1.2 Contract Manager (CM). The CM is the Government agent within the contracting office who performs day-to-day administration of the contract. The CM may also be the CO if the person has a warrant.

NOTE: The CM can be reached at DSN: 574-5706 or Commercial 757-764-5706.

2.2.2 Program Management

2.2.2.1 Program Manager (PM). Executes management and oversight for the delivery of contractually acquired services. Keeps up-to-date on mission changes that may drive the need for a contract modification. Ensures Contract Performance Assessment Reports (CPARS) are conducted. Reviews the COR’s contract performance documentation on a regular basis to ensure performance is compatible within the Requirement Owner’s acceptable risk tolerances in order to meet contract and mission objectives.

NOTE: The PM for this contract is from HQ ACC/TRSS. The PM may be reached at: DSN:

574-4351 or Commercial 757-764-4351

2.2.3 Quality Management

2.2.3.1 Functional Services Manager/COR Management. Ensures continuous contract monitoring by a qualified COR. Review, approve, and submit COR nominations packages to the CO to enable appropriate contract surveillance. Ensures personnel are properly trained and technically qualified to perform COR duties, have no personal conflict of interest, and COR responsibilities are assessed on the member’s performance report/appraisal. Evaluates the individual’s performance as a COR and solicit feedback from the CO to include in the overall evaluation. Evaluation of CORs performance occurs if the COR is dedicated full-time, part time or if the CORs duties are assigned as additional responsibilities. Ensures the COR has the necessary clearance for the contract. Ensures the COR is provided and completes all required annual and three year refresher training, along with any additional training mandated by the contracting officer.

Receives contract specific training from the QAPC.

2.2.3.2 Quality Assurance Program Coordinator (QAPC). The QAPC integrates the quality contract requirements into the quality assurance program and acts as the liason between the contracting and functional organizations; specifically reviewing the PWS and QASP to ensure that requirements are clearly stated, enforceable, and measureable. Trains CORs and COR Management (e.g., COR Supervisor, FSM) on the contracting requirements associated with the quality assurance program and MAJCOM/DRU procedures. Tracks all contract nonconformances, assisting CORs in drafting nonconformances notices (1st/2nd notice/CAR).

NOTE: The QAPC can be reached at DSN: 574-9365 or Commercial 757-764-9365.

2.2.3.3 Chief, Contracting Officer’s Representative (C-COR). The C-COR is the team lead for all COR surveillance activities on a single contracted effort. As such, the C-COR will develop and publish a schedule of those surveillance activities; receive and consolidate surveillance reports from other CORs for submission to the PM/FSM and CO. The C-COR will maintain the central non-conformance log for all CORs on the contract and submit non-conformance reports IAW this QASP. The C-COR will be assigned as an Assessing Official’s Representative (AOR) in the Contractor’s Performance Assessment Reporting System (CPARS) and will submit a draft evaluation IAW HQ ACC CPARS Policy. The C-COR must meet the experience and training requirements for CORs, as found in paragraph 2.2.7.

NOTE: No C-COR is assigned to this contract. All C-COR duties referred to throughout this document will be completed by the COR.

2.2.3.4 Contracting Officer Representative (COR). A COR is responsible for implementing all QA matters and procedures applicable to this award. COR(s) should be at a level where other duties don’t prevent proper oversight of Contractor performance. The COR must maintain both technical competency and evaluation proficiency in Contractor surveillance procedures. The COR must also have enough knowledge of Contractor activities to properly review and disposition monthly invoices. The COR cannot assume duties until nominated by the requiring agency, completes all required COR training (DAU, Ethics, CTIP, and CO/QAPC or CO led training), and is designated in writing by the CO. The COR is responsible for overall surveillance to include surveillance conducted by Subject Matter Experts (SMEs). If SMEs are used, the COR will develop and implement a plan to collect SME surveillance documentation for consolidation into the monthly COR Surveillance report. If a SME identifies any areas of concern, the COR will verify the performance issue and document it IAW paragraph 4.1.

2.2.3.5 Subject Matter Expert (SME). A SME is a technical or functional area expert that may be used to assist CORs surveil contractor performance. If used, a SME will be responsible for validating Contractor performance within the area of their expertise, only. The SME will receive contract-specific training from the C-COR, be appointed in writing, and be responsible for providing and/or reviewing documentation to/for the COR.

2.2.3.5.1 SMEs are responsible for monitoring contractor performance in the SME’s respective area of expertise and providing documentation, as required by DoDI 5000.72, DFARS PGI 201.602, AFFARS MP 5301.602-2(d), and this instruction, to the C-COR.

2.2.3.5.2 SMEs are nominated by their chain of command, then trained and approved by the C-COR. When no longer required or able to perform SME duties (due to PCS, change of duties, etc.), the SME must be formally relieved of duties by the office which submitted the original nomination, or by the COR. SMEs are technical experts and can only recommend changes to a service or supply when a technical error is discovered. Any technical error discovered by a SME should be resolved at the lowest level possible. Any disputes between contractor and SME regarding technical errors will be elevated to the COR immediately but NLT 5 business days.

2.2.3.5.3 SMEs shall receive training conducted by the C-COR prior to assuming SME duties. Training will, at a minimum, consist of functional SME responsibilities, ethics, and review of applicable regulations along with a detailed discussion of the contract.

2.2.3.6 Government Flight Representative (GFR) and Government Ground Representative (GGR). GFRs and GGRs maintain surveillance of flight and ground operations and perform duties prescribed by AFI 10-220, Contractor’s Flight and Ground Operations. GFRs and GGRs must be Defense Contract Management Agency (DCMA) trained to surveil the Contractor using the criteria outlined in the PWS and AFI 10-220. The GFR/GGRs is attached to the 552 ACW, with dual responsibilities as a GFR and an Instructor Pilot. GFRs and GGRs are not CORs and are not subject to the same training, experience, and reporting requirements.

2.2.3.7 Additional Surveillance Roles. The contractor may be evaluated by 12th AF and 552 Wing personnel while conducting annual Safety and Stan/Eval Staff Assistance Visit(s) in conjunction with the 552 ACW IG Unit Effectiveness Inspection. The contractor may undergo in-flight evaluations and ground testing of their crewmembers. Results of these evaluations will be provided to the COR and handled IAW this QASP.

2.3 Contractor Key Personnel.

2.3.1 Program Manager. The Contractor’s PM is responsible for the overall execution of the contract and has the full authority to act on behalf of the Contractor. This individual will serve as the focal point for the CO, CM, Government PM and COR assigned to the contract and may interface with the customer, as needed. The PM for this contract is Mr./Mrs. First Last, Company Name, email at email@corporate.com.

2.3.2 Site Manager. The Site Manager is responsible for technical or quality control oversight, and day to day operations of the contract. The SM for this contract is Mr./Mrs. First Last, Company Name, email at email@corporate.com.

3.0 CONTRACT SURVEILLANCE

The COR is responsible for a wide range of surveillance requirements that effectively measure and evaluate Contractor performance. The Contractor, not the Government, is responsible for contract performance management and Quality Control (QC) actions to meet the terms of the contract.

However, because of the commercial nature and criticality of the work performed by Contractors, the Government cannot rely solely on the Contractor’s quality program but must have a method to oversee performance of the entire contract. The Government does this through the QASP and can inspect or test all services called for by the contract, to the extent practicable, at all times and places, during the term of the contract.

3.1 Surveillance Methods.

Four methods of surveillance will be used: Initial Performance Assessment (IPA), One Hundred Percent Inspection, Periodic Audits, and Customer Feedback. Other methods of surveillance must be fully documented, reviewed by the QAPC and approved by the CO before implementation.

Information obtained from these activities may be used by the PM/COR/Requirement Owner to ascertain whether or not Contractor performance is compatible with contract and mission objectives and by the CO for actions relating to the Contractor.

3.1.1 IPA. The initial evaluation of contractor performance is a joint determination by the MFT that may include CO, CM, PM, QAPC, CORs, and SMEs to determine if the Contractor completed transition, successfully started performance, is fully operational, and is within the estimated cost, schedule, and performance parameters of the contract. The review shall take place following transition, but no earlier than (NET) 30 days after the Contractor assumes full performance responsibility. The COR will complete the report received from the CM, then return it to the CM.

3.1.2 One Hundred Percent Inspection. This method will be used for Contractor submissions, products, and deliverables (tangible items). CORs/SMEs will use the One Hundred Percent Inspection method to evaluate all reports delivered under paragraph 3.2. The COR, with assistance from the CM and CO if necessary, accepts or rejects these products based on the criteria and performance standards as outlined in the Services Summary. The COR also documents all results of inspections actions, to include SME surveillance of tangible items, and subsequent Contractor re-performance taskings, if applicable.

3.1.3 Periodic Audit. This type of surveillance consists of COR evaluating products/services not surveilled via One Hundred Percent Inspection or Customer Feedback. Periodic Audits are implemented to meet COR-defined reporting requirements. Contract terms not outlined in the Service Summary and not routinely scheduled for inspection shall be subject to Periodic Audit mailto:email@corporate.com mailto:email@corporate.com throughout the performance period. The COR shall only surveill items as scheduled and approved by the Government PM.

3.1.4 Customer Feedback. The COR will receive and validate Customer Feedback as a surveillance method. Validation of Customer Feedback may consist of reviewing Contractor-provided deliverables, invoices, and contacting points of contact at, or near, the point of delivery. No action will be taken based on Customer Feedback without prior validation. Additionally, the COR and SMEs will use the contractor-provided end of course surveys as a surveillance method and the information provided in them to improve future training materials.

3.2 Surveillance Documents

3.2.1 Deliverables. All deliverables will be reviewed for timeliness, accuracy and format. If a deliverable due date falls on a weekend or holiday, the Contractor shall submit the deliverable on the last business day prior to the due date.

Table 1 – Deliverables

DELIVERABLE DUE DATE Frequency DISTRIBUTION REFERENCE Training Workload Management Plan (TWMP) With Contractor Proposal One Time CO/COR PWS para 1.8., 4.8.1.

Instructional Systems Development Management Plan (ISDMP)

NLT 10 days after contract award;

Within 14 calendar days of notification to modify

One Time;

As required

COR/SME

PWS para 1.10.2., 4.8.2., Attachment

Instructor Training and Certification Plan (ITCP) With Contractor Proposal One Time CO/COR PWS para 4.8.3.

Quality Program Plan With Contractor Proposal, under ISDMP One Time CO/COR

PWS para 4.8.2.(7), 4.8.4., 4.7.17

Strike Mitigation Plan NLT 10 days following contract award One Time COR PWS para 4.8.5

Transition Plan With Contractor Proposal One Time CO/COR PWS para 4.8.6.

Courseware Development Schedule (CDS)

Initially, NLT 30 days after beginning of transition;

Thereafter, NLT 15 calendar days prior to the beginning of each quarter of effectiveness

During transition, then quarterly TRSS Det 6/CC PWS para 1.10.4., Style Guide A1

Unit Lesson Strategy Report As approved by the

Government in Contractor’s

CDS

As approved by the Government in

Contractor’s CDS

SME

PWS para 1.9.2.2., 4.11.43., Appendix

D para D.1.3.2., Style Guide A2

Lesson Overview As approved by the

Government in Contractor’s

CDS

As approved by the Government in

Contractor’s CDS SME Style Guide A3

Instructor Media Package As approved by the

Government in Contractor’s

CDS

As approved by the Government in

Contractor’s CDS SME Style Guide A4

Lesson Package As approved by the

Government in Contractor’s

CDS

As approved by the Government in

Contractor’s CDS SME PWS para 4.11.27., Style Guide A5

Lesson Plan As approved by the

Government in Contractor’s

CDS

As approved by the Government in

Contractor’s CDS SME Style Guide A6

Lesson Study Guide As approved by the

Government in Contractor’s

CDS

As approved by the Government in

Contractor’s CDS SME Style Guide A7

Test Package As approved by the

Government in Contractor’s

CDS

As approved by the Government in

Contractor’s CDS SME PWS para 1.9.5.3., Style Guide A8

Training Evaluation Document

As approved by the Government in Contractor’s

CDS for each class;

Annual due NLT end of each FY

As approved by the Government in

Contractor’s CDS;

Annually

SME Style Guide A9

Contractor’s Progress, Status, and Management Report

NLT 5th working day of the month Monthly Det 6/CC, COR, CM PWS Attachment 1

Conference/Meeting Minutes Within 5 business days of the conference/meeting As required Det 6/CC and all attendees PWS para 1.13., Style Guide A11

Course Syllabus

As approved by the

CDS for each course;

As approved by the

SME Style Guide A12

Training Task Lists

As approved by the

As approved by the

SME Style Guide A13

Syllabus Review Report

As approved by the

As approved by the

SME Style Guide A15

Courseware Revision Summary Report

As approved by the

CDS;

As approved by the

SME Style Guide A16

Crtierion Referenced Objective Document

As approved by the

As approved by the

SME Style Guide A17

Student Handbook

As approved by the

As approved by the

SME Style Guide A18

Positional Grade Sheet

As approved by the

As approved by the

SME Style Guide A19

GFE Inventory

NLT 15 calendar days from contract start;

Annually thereafter

Annually SME PWS para 3.3.1

Test Bank Master File NLT end of each FY Annually SME PWS para 1.10.3.1

Employee Listing

At performance start;

Updated per employee status/info change;

NLT start of new Option

Year

Initial;

As Required;

Annually

Unit Security Office/COR PWS para 4.6.5

3.2.2 Services Summary. The Services Summary (SS) is a list of critical tasks that must be performed by the Contractor, the performance measure for those tasks, and the method of surveillance the COR/SME will use to validate/inspect these tasks. These items must be assessed/evaluated with results documented in the COR folder and the AMS Monthly COR Report. The table below (Table 2) identifies the Service Summary for the contract and the associated method of surveillance for each performance objective.

Table 2 – Services Summary

PERFORMANCE

OBJECTIVE

PWS Para PERFORMANCE MEASURE with Acceptable Quality Level [AQL]

Method of Surveillance

Contractor maintains qualified personnel.

Para 4.8.3 / 5.4, A5.3-5, Appendix F.

Contractor provides qualified personnel for all required positions at performance start (PWS 5.4., A5.3-A5.5.) and effectively trains and certifies 100% of personnel IAW its Instructor Training and Certification Plan (PWS 4.8.3.) and current instructions to meet the USAF performance standards.

AQL: 100%

See Attachment

Contractor performs academic instruction

1.7.2., 1.7.3., Appendix C

Contractor ensures qualified instructors attend all academic events and conduct the events as required (PWS 1.7.2., 1.7.3).

Contractor provides proficient mission crew members for all crew positions, qualified IAW current instructions and guidance to meet the USAF performance standards.

AQL: 100%

Contractor performs Training Device Instruction.

1.7.4., Appendix C

Contractor ensures qualified instructors attend all Training Device events and conduct the events as required (PWS 1.7.4.).

Contractor provides proficient mission crew members for all crew positions, qualified IAW current instructions and guidance to meet the USAF performance standards.

AQL: 100%

Contractor performs Flight Instruction.

Appendix F Contractor ensures qualified instructors attend all Flight Instruction events and conduct the events as required (PWS Appendix F.).

Contractor provides proficient crew members for all crew positions, qualified IAW current instructions (F5.8.3.5.) and guidance to meet the USAF performance standards.

AQL: 100%

Contractor develops and delivers courseware.

1.9., Appendix D

Contractor develops all courseware IAW PWS 1.9. & Appendix D and the contractor’s ISDMP.

All courseware outlined in the CDS is delivered by 31 January, annually.

Contractor provides ~25% (+/- 5%) of courseware to be developed on a quarterly basis.

All delivery schedule deviations are coordinated throught the COR.

AQL: 100%

Contractor develops, maintains, updates, reviews and delivers MSTP for generation of AWACS events

Para 1.12.6., Appendix E

Contractors shall ensure compatibility with AWACS aircraft software and version modifications and to allow MSTP operation of all simulators IAW

PWS 1.12.6.

AQL = 90%

Contractor meets all security requirements.

4.6. Contractor ensures all employees adhere to 100% of the security requirements addressed in the PWS.

AQL: 100%

Contractor delivers required data, reports, and other deliverables.

Para 1.12.1 Attachment

Contractor delivers 100% of reports, schedules, and deliverables IAW approved formats, as specified in this PWS.

The contractor ensures critical data is accurately entered, access is controlled, and available to appropriately cleared Government personnel.

AQL: 100%

Contractor performs academic scheduling and registrar support

PWS Para

1.12.1 / 1.12.2

Contractor shall ensure accuracy, and timely dissemination of all student management system products and records.

Contractor shall ensure no interruption to scheduled training events for aircrew student qualifications.

AQL=100%

3.2.3 Service Summary Performance Objectives Checklist. The COR is responsible for developing/updating standard procedures for surveillance checklists and may request outside assistance. Changes to the surveillance checklists (modifying, adding or removing) require the QASP be reviewed in its entirety by the offices identified on the title page. The SS Performance Objective Checklists serve as the guideline for all inspections (see Attachment 4).

3.2.4 Contractor’s Quality P Plan (QCP). A Contractor's internal actions, in the form of a QCP, or as in this contract, a Quality Program Plan (QPP) , are written to satisfy contract requirements for written procedures or to provide direction on how Contractor employees are to perform PWS tasks.

Any deviations will be treated as deficiencies/non-conformances and handled IAW paragraph 4.1.

The QAPC, PM, and COR will review the QCP IAW requirements of the solicitation and subsequent awarded contract. The COR will forward any deficiencies/non-conformances to the QAPC and CO/CM for appropriate action.

3.2.5 SME Assessment. SMEs will have routine interaction with the contractor during the performance of PWS tasks. When a SME identifies technical or funcational inaccuracies, they will notify the contract employee that provided the material or service in error and the C-COR immediately but NLT five (5) business days of the discovery. SMEs will review, and be familiar with, self-inspection procedures in the Contractor’s QCP. The SME is responsible for providing monthly, quarterly, and annual documentation to the C-COR for consolidation and submission in one report to the CO and PM.

3.2.6 Assessment. The normal service summary performance assessment period for CORs is monthly, by contract year, however, the SME will ensure all service summary performance items are assessed at least quarterly. For this contract, the quarters are: Feb – Apr, May – Jul, Aug – Oct and Nov – Jan.

3.3 Deadline to Submit Report.

3.3.1 SS Performance - Submit the SS (Table 2) to the C-COR within five (5) business days after the end of the calendar month. All items in the SS do not need to be reviewed each month, but all items will be reviewed at least once, quarterly. Each COR will forward the completed SS to the C- COR. Reports may be submitted electronically.

3.3.2 Contract Discrepancy – Any contract discrepancy discovered by a COR or SME must be brought to the C-COR’s attention by email within five (5) business days. See Table 3 for types of discrepancy and paragraph 4.1 for non-conformance reporting.

3.3.3 The COR is authorized to perform inspection/surveillance at any time during contract performance IAW FAR 52.212-4 and 52.246-4. CORs will assess contractor SS performance on a monthly basis. All functional areas within the PWS will be inspected by the COR at least once per year and will include regular reviews of the contractor's QPP to ensure contractor's compliance with it, as required by FAR 52.246-11.

3.4 Additional COR Inspections.

3.4.1 Inspect Contractor’s Quality Program Plan (QPP). A Contractor's internal actions, in the form of a QPP, are written procedures for inspections, records, and security procedures or to provide direction on how Contractor employees are to perform. The COR will review the QPP IAW requirements of the solicitation and subsequent awarded contract, validating the contractor is keeping, and making available to the Government, complete records of its inspection work. Any deviations will be treated as deficiencies/non-conformances and handled IAW paragraph 4.1 The Contractor's QPP is contained in the Instructional Systems Development (ISD) Management Plan.

3.4.2 Evaluate Combating Trafficking In Persons (CTIP). The COR and SMEs will monitor the Contractor’s performance regarding trafficking in persons such that non-compliance with FAR Clause 52.222-50, Combating Trafficking in Persons, is brought to the immediate attention of the CO.

3.4.2.1 Violations and remedies.

3.4.2.1.1 If the contracting officer receives information via the Monthly COR Report or other method indicating that the contractor, a contractor employee, a subcontractor, or a subcontractor employee has failed to comply with the requirements of the clause at FAR 52- 222-50, the contracting officer shall,

3.4.2.1.2 Through the contracting officer’s local commander or other designated representative, immediately notify the Combatant Commander responsible for the geographical area in which the incident has occurred; and

3.4.2.1.3 Information for any investigation and enforcement may be made to:

DoD Inspector General Hotline at http://www.dodig.mil/hotline or 800-424-9098, CTIP Program Manager on-line at http://ctip.defense.gov or by email to ctipreports@osd.pentagon.mil, National CTIP Hotline at 1-888-373-7888, or, CTIP Hotline website at http://www.polarisproject.org.

3.4.2.1.4 Issue a non-conformance.

3.4.3 Perform an IPA. The MFT will conduct a review of Contractor’s performance NET 30 days after the contract start date. SMEs and CORs will submit IPA results to the C-COR for consolidation.

The C-COR will forward the IPA (Attachment 2) to the CM and QAPC.

4.0 CONTRACTOR PERFORMANCE REPORTING

4.1 Non-conformances

Audit or Inspection results that fall below contractual standards shall be identified as non-conformances.

4.1.1 Types of Non-conformance.

4.1.1.1 Minor Non-conformance. A minor non-conformance is a non-conformance, which by itself does not adversely impact mission, safety of personnel and/or equipment, performance (quality), schedule (delivery), or cost. A minor non-conformance normally does not increase risk to the Government.

4.1.1.2 Major Non-conformance. A major non-conformance in this QASP meets the FAR

46.101 definition of both a “Critical nonconformance” and a “Major nonconformance.” It is a non-conformance that adversely impacts, up to preventing performance of, a vital agency mission, safety of personnel and/or equipment, performance (quality), schedule (delivery), or cost. A http://www.dodig.mil/hotline http://ctip.defense.gov/ mailto:ctipreports@osd.pentagon.mil http://www.polarisproject.org/ major non-conformance also increases risk to the Government. An example of increasing risk would be significant number of recurring non-conformances, which indicates inadequate preventive measures or actions, thus lowering the Government’s confidence that quality services will be provided on time and at cost.

4.1.1.3 Non-conformance Notification. If a deficiency is found during inspection of the Contractor’s performance, the C-COR will determine the type of notification based on the category of the non-conformance (see Table 3). Non-conformances pre-identified by the Contractor through the Contractor’s QCP/internal audit does not require a govnerment non-conformance notice; however, if the Contractor fails to take corrective actions in a reasonable time-frame, the C-COR will notify the PM, CO and QAPC for further action.

Table 3 – Non-conformance notifications

Non-conformance Category Notification Type Minor

Step 1: First Notice Step 2: Second Notice

Major Corrective Action Request (CAR)

4.1.1.4 First or Second Notices. As shown in Table 3, First and Second Notices provide written notification from the Government to the Contractor of minor non-conformances. Initial minor non-conformances should be addressed with a First Notice to the applicable Contractor representative. The C-COR will notify the FSM, CO, and QAPC (ACCAMICQAPC@us.af.mil) of the issue, and prepare the notice for CO signature.

4.1.1.4.1 The C-COR will coordinate with the QAPC on the first or second notice NLT five

(5) business days after non-conformance notification, ensuring the first or second notice (Attachment 1) is filled out correctly. The QAPC will provide an electronic copy of the coordinated first or second notice to the CO NLT 10 duty days after discovery of the event. If deemed appropriate, the CO will issue the First or Second notice to the contractor with a suspense date for the contractors reply, sending a courtesy copy email to both the QAPC and

C-COR.

4.1.1.4.2 The contractor shall send the completed first or second notice to the CO and QAPC (ACCAMICQAPC@us.af.mil). If the Contractor fails to respond by the specified due date, the QAPC will notify the CO for action and notify the C-COR to annotate the Contractor’s non-responsiveness in the AMS monthly services summary report. If the Contractor fails to resolve the issue as documented in the Notice, the CO may elevate the issue to the next level (Second Notice or CAR). Corrective/preventive actions initiated and/or completed by the Contractor will be verified by the C-COR on scheduled subsequent inspection and documented accordingly.

4.1.1.5 Corrective Action Requests (CAR) (see Attachment 1). As shown in Table 3, a CAR provides written notification from the Government to the Contractor of significant performance discrepancies. The C-COR will recommend to the FSM, CO, and QAPC that a CAR be issued when previous notifications failed to result in corrective actions or when severe deficiencies exist.

If a CAR is deemed necessary, the QAPC will assist the C-COR in filling out the CAR. The draft CAR will be routed NLT 10 duty days after the event to the CO for review and signature, as the CO is the only person who has authority to issue a CAR to the Contractor.

mailto:ACCAMICQAPC@us.af.mil mailto:ACCAMICQAPC@us.af.mil

4.1.1.5.1 To satisfactorily close-out a CAR, the following criteria must be met:

- The Contractor must meet the suspense, and

- The corrective action must have already begun, and

- The Contractor's actions must correct the deficiency, and

- If applicable, be within specified standards.

4.1.1.5.2 If the corrective action is deemed UNSATISFACTORY by the CO, QAPC and C- COR, annotate the CAR accordingly and forward it to the CO with a brief explanation, including an estimate of the Government's loss due to the deficient performance. The estimate may account for loss or destruction of property, supplies, or equipment, and hours lost to re-perform the task. If the task cannot be re-performed, indicate the number of hours it ordinarily would have taken to do the task.

4.1.1.6 The C-COR will maintain a non-conformance log documenting all First Notices, Second Notices and CARs. The log will include the date sent to the Contractor, issue, due date for response and actual response date. If a detailed response is required, include a link to the response and include the response in the applicable COR folder tab.

4.2 COR Monthly Report

The C-COR will complete HQ ACC AMIC’s Acquisition Management System (AMS) Monthly COR Report.

4.2.1 Registration. The C-COR will access AMS via https://cms.acc.af.mil/ and request an account using the following:

4.2.1.1 Organization. Click on link and search for the appropriate organization. If the organization is not in the database, create a new entry by typing in the organization.

4.2.1.2 First Name

4.2.1.3 Last Name

4.2.1.4 Email address. Only use a Government-provided email address that is routinely accessed.

4.2.1.5 User Role. Check the box next to Contracting Officer Representative

4.2.1.6 Justification. Enter, “Primary COR on contract FA4890-19-C-0004.”

4.2.1.7 Click “Submit Information”

4.2.2 The C-COR certifies the essential services accomplished by the contractor and documents any issues or non-conformances that were identified during the month.

4.2.3 AMS will generate monthly emails to the COR(s) designated on the contract, monthly reports will be completed NLT the 10th calendar day of each month.

4.2.4 A COR will click on the embedded hyperlink in the email, or copy and paste the link onto their browser, OR log into AMS, click on My Requirements, then click on appropriate contract; click on Monthly COR Report.

4.2.5 The monthly report will display questions for the previous month (e.g., answering questions in the month of March will display the monthly report for February).

4.2.5.1 Click on Answer Questions

https://cms.acc.af.mil/

4.2.5.2 Click on meets, above, below standards or N/A (see Table 4) and provide justification for each answer.

4.2.5.3 After completing the Monthly COR Report, click submit.

4.2.5.4 AMS will only allow one submitted report per month.

Table 4 – Monthly Performance Rating Definitions

Rating Definition Above Standards

Contractor performance exceeds contractual requirements and is beneficial to the government. To justify exceeds standards, state which PWS or SOW requirement(s) was exceeded (to include the paragraph number) along with how exceeding this requirement was a benefit to the government.

Meet Standards

Contractor performance meets contractual requirements. To justify meets standards, specify the PWS paragraph/requirement and state that the contractor met the requirement.

Below Standards

Contractor performance does NOT meet some contractual requirements. To justify below standard, identify an event(s) that the Contractor had trouble overcoming and state how it impacted the Government. This rating should be supported by referencing any non-conformance notification sent to the Contractor. Under Corrective Action, state what correction action the contractor has taken or is planning to take based on the contractor’s response to the non-conformation notification.

N/A Contractor was not contractually required to perform specific task or specific question is not applicable for evaluated period.

4.3 Annual Contract Performance Assessment Reporting System (CPARS)

An annual CPAR will be written on all Services contracts that exceed total contract value dollar reporting thresholds. The CPARS website, https://www.cpars.gov/, is the Federal Government’s Enterprise Solution for collection of Contractor Past Performance Information (PPI), as required by the FAR. CPARS is a web-enabled application that collects and manages a library of automated Contractor report cards. The C-COR and PM (if different individuals) will be designated as Assessing Official Representatives (AOR) in CPARS. The AOR will use objective evidence from Contractor deliverables, Monthly/Quarterly Reports, 1st/2nd notices, CARs, WAWF invoices and any other validated information provided by CORs or the customer, that could be used for the CPAR assessment. The AOR will send the Draft CPAR to the Assessing Official (AO) within 20 calendar days following the end of the Period of Performance (PoP). The table below (Table 5) is an excerpt from the AMIC CPARS Roles and Responsibilities Policy Letter.

https://www.cpars.gov/

Table 5 – Performance Assessment Report Routine Timelines*

CPAR Role: Timeframe Action in CPARS AOR +20 calendar days following the end of the Period of Performance (PoP)

Draft (Narrative and rating) report and send to CO

AO +40 calendar days following the end of the PoP

Validate narrative/ratingcontract information and send to Contractor Representative

Contractor Representative 60 calendar days Comment and return to CO AO (Contractor Concurs or does not respond)

7 calendar days but not to exceed PoP end +107 days

Completes CPAR

AO (Contractor Non-Concur) RO (Contractor Non-Concur

14 calendar days but not to exceed PoP end +114 days

CO forwards to RO; RO reviews, updates and completes CPARS

*Focal Point will notify AOR, AO, RO or division chief for all exceeded time frames

4.4 Wide Area Workflow (WAWF).

The C-COR will accept or reject all Receiving Reports submitted by the Contactor in WAWF within 7 days of submittal by the Contractor. To access the receiving report, log into WAWF, select the Government, Acceptor, Acceptance Folder tab; then type in the contract and delivery order (contract/task order) number. The Shipment Number column in WAWF is the Receiving Report number.

4.4.1 Validating Receiving Reports. The C-COR will review and validate all WAWF receiving reports (Shipment Number in WAWF) ensuring that:

• The Contractor submitted a combination invoice/receiving report

• The report reflects the correct contract and TO number

• Beginning and ending dates being billed are stated and accurate

• The correct CLIN(s)/SLIN(s) is(are) charged for billing period

• A description of the services is provided (Firm Fixed Price, Labor Hour, Travel, Materials)

• The correct amount is charged, to include applicable handling rates

• Invoiced travel was pre-approved

• Back-up documentation is attached that includes a Contractor invoice with current and cumulative hours (for FFP LOE contracts) and amounts, as applicable

• A breakout of any other charges, as agreed to by the CO or required by the Task Order, is attached in WAWF, or has been provided to the C-COR

4.4.2 Certifying and Accepting Receiving Reports. Inspection of Services is ALWAYS performed by the appointed COR. The C-COR will validate and certify services through non-conformance notifications (para.4.1) and COR monthly reports (para.4.2.2.). After validation of services, the C- COR will select the “CQA” (Contract Quality Assurance) button to verify that the invoice was accurate, this step will normally select the Acceptance button, also. If anyone other than the C-COR is accepting services for the contract, the C-COR MUST certify in writing to the acceptor that services have been inspected/validated, the acceptor must then state in the comments block (under Misc Info tab) that the C-COR has reviewed, validated and certified services/invoices. In all cases, the C-COR will retain documentation (electronic or hard copy) showing COR validation and certification, as well as final acceptance of Contractor invoices.

4.4.3 Expenditure Log. After certification of an invoice in WAWF, the C-COR will update the expenditure log documenting/tracking certified invoice expenditures against funds allocated on the contract against each CLIN. Funding belongs to the organization where the Requirement Owner resides; the expenditure log allows the C-COR to ensure that funds are available for the work performed, to include travel and materials.

4.4.4 Rejecting Receiving Reports. If a receiving report is rejected in WAWF, the C-COR will go to the Miscellaneous Info tab to complete the comments block as follows:

4.4.4.1 List details regarding all unsatisfactory services or incorrect invoice information, and

4.4.4.2 Instruct the Contractor to contact DFAS to have the invoice rejected back to the Contractor; this will allow the Contractor to make the same correction on both the receiving report and invoice, since combination invoices/receiving reports are required.

4.4.4.3 The rejecter will then return to the Header tab and click the “Reject to Initiator” block.

After rejecting a receiving report for unsatisfactory services, e-mail the CM/CO the contract number, receiving report number, and reason for rejection. Rejections for incorrect invoice information will be tracked by the COR.

5.0 COR FOLDER.

The COR will establish and maintain a COR folder for each contract/task order; an electronic version of the folder may be kept, however, it must be maintained in an area where alternate CORs and COR Management will have access to it. The COR folder must contain the tabs and contents outlined below in Table 6. All surveillance records are considered FOR OFFICIAL USE ONLY documents, and must be marked and safeguarded as such. Documentation is kept for the life of the contract and provides necessary continuity should COR duties transition during the course of the contract. COR Management and the COR will receive instructions from the CO on disposition or archival procedures upon contract completion.

Table 6 – COR Folder Tabs & Contents

Tab Title Items to Include 1 General Correspondence If electronic, must be available upon request and for the life of the contract 2 COR/SME designation Letter(s) Assignment/nomination letter(s), designation letter(s), copy of termination letters, and Government/Contractor relationships guidance

3 COR/SME Training COR/SME certificates for DAU, Ethics, CTIP, and CO/QAPC or CO Led training

4 Contract/Task Order, Modifications, PWS, Pricing

Contract for current level of effort, any modifications and most current PWS/amendment on contract, and Contractor’s contract/task order pricing

5 Quality Assurance Surveillance Plan, Surveillance Checklist

Contract/task order QASP, approved surveillance checklist, and completed surveillance checklists

6 All Required Contractor Plans Training Workload Management Plan ISD Management Plan Instructional Training and Certification Plan Quality Program Plan Strike Mitigation Plan Transition Plan

7 First & Second Notices, Corrective Action Reports (CAR)

Non-conformance log, First and Second Notices and CARs issued to Contractor

8 Contractor Monthly Reports 9 WAWF Invoices WAWF invoices can be maintained within WAWF, however, WAWF attachments must be inspected and maintained in Tab 9.

10 COR Expenditure Log Updated with latest invoice and funding balance 11 DD Form 254 If applicable 12 Monthly Log The COR will maintain a written or electronic log which can be used to assist in tracking non-conformances and any other information the COR deems necessary or appropriate.

13 CPARS All documentation used to develop a CPAR

Attachment 1 – Non-Conformance Report (Example Only)

QMSF 8.3-1

Non-Conformance F

Attachment 2 – Initial Performance Assessment (IPA) (Example Only)

MEMORANDUM FOR ACC AMIC/PKB Date

FROM: (COR’s Office)

SUBJECT: Initial Performance Assessment for E-3 Contract Aircrew Training and Courseware Development on Contract FA4890-20-C-####

Contract Dollar Value: $

1. Per the contract Quality Assurance Surveillance Plan, an initial assessment of Contractor performance is due no earlier than 30 days after the Contractor assumes full performance responsibility (i.e., after completion of transition). The IPA is a joint determination by the Multi- Functional Team that the Contractor has successfully started performance, completed transition, is fully operational, and is within the estimated cost, schedule, and performance parameters of the contract. The CO may waive the initial evaluation for Contractors that have continued performance under a successor contract award (prior incumbent), or for Contractors that have otherwise demonstrated full compliance with contract start-up. For acquisitions greater than or equal to $100M, report IPA results to the PM.

2. As the COR, please complete this assessment and return it to the Contract Manager (CM) administering the contract. Using the Contract Performance Assessment Ratings Legend below, enter a rating in the Rating Column of the table below.

3. For the IPA Assessment:

Contract Performance Assessment Rating Successfully Started Performance Completed Transition Fully Operational Within Cost Within Schedule Within Performance Parameters of Contract Management (include for acquisitions =/> $100M) Technical (include for acquisitions =/> $100M)

Contract Performance Assessment Ratings Legend G = Green – No issues.

Y = Yellow – Issue(s), but Contractor has an adequate mitigation or corrective action plan in place.

R = Red – issue(s) with inadequate or no Contractor mitigation or corrective action plan.

Any “red” assessment shall include the Government-proposed actions for the failing Contractor.

4. Negative variations in cost, schedule, staffing, and performance are reported below with an assessment of the root causes and…