DRAFT_C2_QASP_v_0_3_Sep_19.pdf

Quality Assurance Surveillance Plan # Mmm 19, Rev Basic C2 CAT/CWD Solicitation # FA4890-20-R-0003

BY ORDER OF THE DIRECTOR HQ ACC ACQUISITION

MANAGEMENT AND

INTEGRATION CENTER

## Mmmmmm 2019 Revision date – Basic

Quality Assurance Surveillance Plan

Command and Control (C2) Contract

Aircrew Training (CAT) and Courseware Development (CWD)

Solicitation: FA4890-19-R-A025 Contract: FA4890-20-C-####

Pages: 45

COMPLIANCE WITH THIS DOCUMENT IS MANDATORY

NOTICE: This Document is available in digital format through ACC AMIC/DRQQ

OPR: ACC TRSS/ACQ Reviewed by: ACC TRSS/ACQ

STEPHEN J. SATAVA, GS-13, DAF RAYMOND L. CHURCHILL, GS-14, DAF

Program Manager Chief, Training Acquisition Flight

Reviewed by: ACC AMIC/DRQ Accepted by: ACC AMIC/PKB

JOHN D. NOBLE, NH-03, DAF KELLI A. WRIGHT, NH-03, DAF

Quality Assurance Program Coordinator Contracting Officer

FOR OFFICIAL USE ONLY (when filled in)

Contents

1.0 SCOPE

1.1 Authority

1.2 Purpose

1.3 Related Documents

2.0 MANAGEMENT LOCATIONS AND KEY PERSONNEL

2.1 Locations

2.2 Government Multi-Functional Team

2.3 Contractor Key Personnel

3.0 CONTRACT SURVEILLANCE

3.1 Surveillance Methods

3.2 Surveillance Documents

3.3 Deadline to Submit Report

3.4 Additional Inspections

4.0 CONTRACTOR PERFORMANCE REPORTING

4.1 Non-conformances

4.2 COR Monthly Report

4.3 Annual Contract Performance Assessment Reporting System (CPARS)

4.4 Wide Area Workflow (WAWF)

5.0 COR FOLDER

5.1 SME Files

Attachment 1 – Non-Conformance Report (Example Only) Attachment 2 – Initial Performance Assessment (IPA) (Example Only) Attachment 3 – Quarterly Surveillance Report Attachment 4 – Individual Performance Objective Inspection Checklists

SS PERFORMANCE OBJECTIVE # 1

SS PERFORMANCE OBJECTIVE # 2

SS PERFORMANCE OBJECTIVE # 3

SS PERFORMANCE OBJECTIVE # 4

SS PERFORMANCE OBJECTIVE # 5

SS PERFORMANCE OBJECTIVE # 6

SS PERFORMANCE OBJECTIVE # 7

SS PERFORMANCE OBJECTIVE # 8

SS PERFORMANCE OBJECTIVE # 9

Attachment 5 – Sample SME Contract Monitoring and Surveillance Report Attachment 6 – Sample SME Nomination Letter Attachment 7 – Sample SME Relief from Duties Letter Attachment 8 – Sample Combination SME Nomination & Relief from Duties Letter

FOR OFFICIAL USE ONLY (when filled in)

Attachment 9 – Guidelines For Courseware Deliverable Evaluation

TABLES

Table 1 – Deliverables Table 2 – Services Summary Table 3 – Non-conformance notifications Table 4 – Monthly Performance Rating Definitions Table 5 – Performance Assessment Report Routine Timelines* Table 6 – COR Folder Tabs & Contents Table 7 – SME File Tabs & Contents

FOR OFFICIAL USE ONLY (when filled in)

1.0 SCOPE

The Quality Assurance Surveillance Plan (QASP) is applicable to personnel performing contract surveillance inspections in accordance with Federal Acquisition Regulation (FAR) Part 46, Quality Assurance; Department of Defense Instruction (DODI) 5000.72 DoD Standard for Contracting Officer’s Representative (COR) Certification; DOD FAR Supplement (DFARS) Subpart and PGI 201.602-2 , Contracting Officer Responsibilities; Air Force FAR Supplement (AFFARS) Mandatory Procedures (MP) 5301.602-2(d) Designation, Assignment, and Responsibilities of a COR; Air Force Instruction (AFI) 63-138, Acquistion of Services; and organizational policy. It is designed to provide the Contracting Officer Representative (COR) a systematic surveillance method for each service identified in the Performance Work Statement (PWS). The Program Manager (PM)/Functional Service Manager (FSM) and COR develop the methods for administering and evaluating performance. The Quality Assurance Program Coordinator (QAPC) and Contracting Officer (CO) or Contract Manager (CM), will review these methods, which are then accepted or rejected by the CO.

1.1 Authority

1.1.1 The following FAR provisions and clauses provide authority for this QASP.

1.1.2 Applicable inspection clauses (not all inclusive, see contract for all clauses):

FAR 52.212-4 – Contract Terms and Conditions -- Commercial Items

1.1.3 Applicable FAR Provision information (not all inclusive, see contract for all provisions):

1.1.3.1 FAR 12.301(b)(3) authorizes the Contracting Officer to tailor Clause 52.212-4 in accordance with FAR 12.302 -- Tailoring of Provisions and Clauses for the Acquisition of Commercial Items. In the case of this contract, the PWS is tailored to institute 52.246-4 and require the Contractor to provide a Quality Control Plan as a part of their Management Plan.

1.1.3.2 FAR 46.202-1, Contracts for Commercial Items. This provision places the responsibility for production quality assurance on the contractor. Any in-process audits conducted by the Government must be consistent with commercial practices.

1.1.3.3 FAR 46.202-3, Standard Inspection Requirements. This provision authorizes the use of clause 52.246-4 and require the contractor to provide and maintain an inspection system acceptable to the Government, and make the inspections records available to the Government.

Additionally, this provision gives the Government the right to inspect services while work is in process.

1.1.3.4 FAR 46.202-4, Higher-Level Contract Quality Requirements. This provision, in conjunction with FAR 46.203 and 46.311, authorize the Government to impose the Contractor use a higher-level quality system like ISO 9001 or SAE AS9100. In addition to in-process audits, the COR may audit the Contractor’s use of and the effectiveness of their quality management system.

1.1.3.5 FAR 46.401, Government Contract Quality Assurance. This provision directs that Government contract quality assurance shall be performed as necessary to determine that the supplies or services conform to contract requirements. Quality assurance surveillance plans will be prepared, and should specify the work requiring surveillance and the method of surveillance.

1.1.3.6 FAR 46.407 -- Nonconforming Supplies or Services. This provision specifies the contracting officer rejects supplies or services not conforming in all respects to contract requirements. The contracting officer ordinarily must give the contractor an opportunity to

FOR OFFICIAL USE ONLY (when filled in) correct or replace nonconforming supplies or services when this can be accomplished within the required delivery schedule; correction or replacement must be without additional cost to the Government.

1.1.3.7 FAR 46.501 – Acceptance (General). Acceptance constitutes acknowledgment that the supplies or services conform with applicable contract quality and quantity requirements, except as provided in this subpart and subject to other terms and conditions of the contract. Acceptance may take place before delivery, at the time of delivery, or after delivery, depending on the provisions of the terms and conditions of the contract. Supplies or services shall ordinarily not be accepted before completion of Government contract quality assurance actions. Acceptance shall ordinarily be evidenced by execution of an acceptance certificate on an inspection or receiving report form or commercial shipping document/packing list.

1.2 Purpose

This QASP provides a systematic method to assess and evaluate the services received from the Contractor, however, this QASP is not considered part of the contract. It defines the policy, procedures, and associated methods used by the Multi-Functional Team (MFT) for planning, preparing, performing, analyzing, and reporting Contractor performance. In the event this QASP conflicts with the contract or task order, the contract or task order shall take precedence.

1.3 Related Documents

1.3.1 C2 CAT/CWD, Contract number FA4890-20-C-####.

1.3.2 Solicition number FA4890-20-R-0003.

2.0 MANAGEMENT LOCATIONS AND KEY PERSONNEL

2.1 Locations

Contract Management and Quality Oversight ACC AMIC/PKB and DRQ 129 Andrews Blvd (Bldg 664) Joint Base Langley-Eustis VA 23665

Program and COR Management

ACC TRSS/ACQ

130 Andrews St., STE 118 Joint Base Langley-Eustis VA 23665

2.2 Government Multi-Functional Team

2.2.1 Contract Management

2.2.1.1 Contracting Officer (CO). CO duties are detailed in FAR subpart 1.602-2, Contracting Officer Responsibilities, and AFFARS MP5346.103, Contracting Officer Responsibilities. The CO is the only Government agent authorized to award or modify contracts; therefore, the CO is the only person authorized to contractually obligate the Government. The CO is also the individual who will sign any notifications to Contractors regarding performance issues and accepts Government surveillance methods used under the award. The CO approves/appoints all CORs assigned by COR Management.

NOTE: The CO can be reached at DSN: 575-1858 or Commercial 757-765-1858

FOR OFFICIAL USE ONLY (when filled in)

2.2.1.2 Contract Manager (CM). The CM is the Government agent within the contracting office who performs day-to-day administration of the contract. The CM may also be the CO if the person has a warrant.

NOTE: The CM can be reached at DSN: 575-4540 or Commercial 757-765-4540

2.2.2 Program Management

2.2.2.1 Program Manager (PM). Executes management and oversight for the delivery of contractually acquired services. Keeps up-to-date on mission changes that may drive the need for a contract modification. Ensures Contract Performance Assessment Reports (CPARS) are conducted. Reviews the COR’s contract performance documentation on a regular basis to ensure performance is compatible within the Requirement Owner’s acceptable risk tolerances in order to meet contract and mission objectives.

NOTE: The PM for this contract is from HQ ACC/TRSS. The PM may be reached at: DSN:

574-4349 or Commercial 757-764-4349

2.2.3 Quality Management

2.2.3.1 Functional Services Manager/COR Management. Ensures continuous contract monitoring by a qualified COR. Review, approve, and submit COR nominations packages to the CO to enable appropriate contract surveillance. Ensures personnel are properly trained and technically qualified to perform COR duties, have no personal conflict of interest, and COR responsibilities are assessed on the member’s performance report/appraisal. Evaluates the individual’s performance as a COR and solicit feedback from the CO to include in the overall evaluation. Evaluation of CORs performance occurs if the COR is dedicated full-time, part time or if the CORs duties are assigned as additional responsibilities. Ensures the COR has the necessary clearance for the contract. Ensures the COR is provided and completes all required annual and three year refresher training, along with any additional training mandated by the contracting officer.

Receives contract specific training from the QAPC.

2.2.3.2 Quality Assurance Program Coordinator (QAPC). The QAPC integrates the quality contract requirements into the quality assurance program and acts as the liason between the contracting and functional organizations; specifically reviewing the PWS and QASP to ensure that requirements are clearly stated, enforceable, and measureable. Trains CORs and COR Management (e.g., COR Supervisor, FSM) on the contracting requirements associated with the quality assurance program and MAJCOM/DRU procedures. Tracks all contract nonconformances, assisting CORs in drafting nonconformances notices (1st/2nd notice/CAR).

NOTE: The QAPC can be reached at DSN: 574-9365 or Commercial 757-764-9365.

2.2.3.3 Chief, Contracting Officer’s Representative (C-COR). The C-COR is the team lead for all COR surveillance activities on a single contracted effort. As such, the C-COR will develop and publish a schedule of those surveillance activities; receive and consolidate surveillance reports from other CORs for submission to the PM/FSM and CO. The C-COR will maintain the central non-conformance log for all CORs on the contract and submit non-conformance reports IAW this QASP. The C-COR will be assigned as an Assessing Official’s Representative (AOR) in the Contractor’s Performance Assessment Reporting System (CPARS) and will submit a draft evaluation IAW HQ ACC CPARS Policy. The C-COR must meet the experience and training requirements for CORs, as found in paragraph 2.2.7.

NOTE: No C-COR is assigned to this contract. All C-COR duties referred to throughout this document will be completed by the COR.

FOR OFFICIAL USE ONLY (when filled in)

2.2.3.4 Contracting Officer Representative (COR). A COR is responsible for implementing all QA matters and procedures applicable to this award. COR(s) should be at a level where other duties don’t prevent proper oversight of Contractor performance. The COR must maintain both technical competency and evaluation proficiency in Contractor surveillance procedures. The COR must also have enough knowledge of Contractor activities to properly review and disposition monthly invoices. The COR cannot assume duties until nominated by the requiring agency, completes all required COR training (DAU, Ethics, CTIP, and CO/QAPC or CO led training), and is designated in writing by the CO. The COR is responsible for overall surveillance to include surveillance conducted by Subject Matter Experts (SMEs). If SMEs are used, the COR will develop and implement a plan to collect SME surveillance documentation for consolidation into the monthly COR Surveillance report. If a SME identifies any areas of concern, the COR will verify the performance issue and document it IAW paragraph 4.1.

2.2.3.5 Subject Matter Expert (SME). A SME is a technical or functional area expert that may be used to assist CORs surveil contractor performance. If used, a SME will be responsible for validating Contractor performance within the area of their expertise, only. The SME will receive contract-specific training from the C-COR, be appointed in writing, and be responsible for providing and/or reviewing documentation to/for the COR.

2.2.3.5.1 SMEs are responsible for monitoring contractor performance in the SME’s respective area of expertise and providing documentation, as required by DoDI 5000.72, DFARS PGI 201.602, AFFARS MP 5301.602-2(d), and this instruction, to the C-COR.

2.2.3.5.2 SMEs are nominated by their chain of command, then trained and approved by the C-COR. When no longer required or able to perform SME duties (due to PCS, change of duties, etc.), the SME must be formally relieved of duties by the office which submitted the original nomination, or by the COR. SMEs are technical experts and can only recommend changes to a service or supply when a technical error is discovered. Any technical error discovered by a SME should be resolved at the lowest level possible. Any disputes between contractor and SME regarding technical errors will be elevated to the COR immediately but NLT 5 business days.

2.2.3.5.3 SMEs shall receive training conducted by the C-COR prior to assuming SME duties. Training will, at a minimum, consist of functional SME responsibilities, ethics, and review of applicable regulations along with a detailed discussion of the contract.

2.2.3.6 Additional Surveillance Roles. The contractor may be evaluated by MAJCOM, Wing, and unit personnel while conducting annual Safety, Stan/Eval Staff Assistance Visit(s), or routine Inspector General inspections. Results of these evaluations will be provided to the COR and handled IAW this QASP.

2.3 Contractor Key Personnel.

2.3.1 Program Manager. The Contractor’s PM is responsible for the overall execution of the contract and has the full authority to act on behalf of the Contractor. This individual will serve as the focal point for the CO, CM, Government PM and COR assigned to the contract and may interface with the customer, as needed. The PM for this contract is Mr./Mrs. First Last, Company Name, email at email@corporate.com.

mailto:email@corporate.com

FOR OFFICIAL USE ONLY (when filled in)

2.3.2 Site Manager. The Site Manager is responsible for technical or quality control oversight, and day to day operations of the contract. The SM for this contract is Mr./Mrs. First Last, Company Name, email at email@corporate.com.

3.0 CONTRACT SURVEILLANCE

The COR is responsible for a wide range of surveillance requirements that effectively measure and evaluate Contractor performance. The Contractor, not the Government, is responsible for contract performance management and Quality Control (QC) actions to meet the terms of the contract.

However, because of the commercial nature and criticality of the work performed by Contractors, the Government cannot rely solely on the Contractor’s quality program but must have a method to oversee performance of the entire contract. The Government does this through the QASP and can inspect or test all services called for by the contract, to the extent practicable, at all times and places, during the term of the contract.

3.1 Surveillance Methods.

Four methods of surveillance will be used: Initial Performance Assessment (IPA), One Hundred Percent Inspection, Periodic Audits, and Customer Feedback. Other methods of surveillance must be fully documented, reviewed by the QAPC and approved by the CO before implementation.

Information obtained from these activities may be used by the PM/COR/Requirement Owner to ascertain whether or not Contractor performance is compatible with contract and mission objectives and by the CO for actions relating to the Contractor.

3.1.1 IPA. The initial evaluation of contractor performance is a joint determination by the MFT that may include CO, CM, PM, QAPC, CORs, and SMEs to determine if the Contractor completed transition, successfully started performance, is fully operational, and is within the estimated cost, schedule, and performance parameters of the contract. The review shall take place following transition, but no earlier than (NET) 30 days after the Contractor assumes full performance responsibility. The COR will complete the report received from the CM, then return it to the CM.

3.1.2 One Hundred Percent Inspection. This method will be used for Contractor submissions, products, and deliverables (tangible items). CORs/SMEs will use the One Hundred Percent Inspection method to evaluate all reports delivered under paragraph 3.2. The COR, with assistance from the CM and CO if necessary, accepts or rejects these products based on the criteria and performance standards as outlined in the Services Summary. The COR also documents all results of inspections actions, to include SME surveillance of tangible items, and subsequent Contractor re-performance taskings, if applicable.

3.1.3 Periodic Audit. This type of surveillance consists of COR evaluating products/services not surveilled via One Hundred Percent Inspection or Customer Feedback. Periodic Audits are implemented to meet COR-defined reporting requirements. Contract terms not outlined in the Service Summary and not routinely scheduled for inspection shall be subject to Periodic Audit throughout the performance period. The COR shall only surveill items as scheduled and approved by the Government PM.

3.1.4 Customer Feedback. The COR will receive and validate Customer Feedback as a surveillance method. Validation of Customer Feedback may consist of reviewing Contractor-provided deliverables, invoices, and contacting points of contact at, or near, the point of delivery. No action will be taken based on Customer Feedback without prior validation. Additionally, the COR and SMEs will use the contractor-provided end of course surveys as a surveillance method and the information provided in them to improve future training materials. The COR/SME will also use the mailto:email@corporate.com

FOR OFFICIAL USE ONLY (when filled in)

Graduate Evaluations administered by TRSS Det 9 for feedback on the training and training materials.

3.2 Surveillance Documents

3.2.1 Deliverables. All deliverables will be reviewed for timeliness, accuracy and format. If a deliverable due date falls on a weekend or holiday, the Contractor shall submit the deliverable on the last business day prior to the due date.

FOR OFFICIAL USE ONLY (when filled in)

Table 1 – Deliverables

DELIVERABLE DUE DATE Frequency DISTRO REFERENCE Strike Mitigation Plan NLT 10 business days of Phase-

In start One Time CO PWS para 1.2.

Instructional Systems Development (ISD) Management Plan

NLT 10 business days of Phase- In start

Thereafter, as directed by the Government

One Time;

As required COR/SME PWS para 1.3.2.2.

Syllabus Review Report

As approved by the Government in Contractor’s CDS COR/SME

PWS para 1.3.2.4. & SG A13

Lesson Plan PWS para 1.3.2.4., 1.4.3., 4.4.27, 4.4.32., & SG A6

Lesson Study Guide PWS para 1.3.2.4., 1.4.3., 4.4.27, 4.4.32., & SG A7

Courseware Revision Summary Report

PWS para 1.3.2.4., 1.4.3., 4.4.27., 4.4.32., & SG

A14

Course Syllabus PWS para 1.3.2.4., 1.4.3., & SG A11

Lesson Overview PWS para 1.3.2.4., 4.4.27, 4.4.32. & SG A3

Unit Lesson Strategy Report PWS para 1.3.2.4.,

4.4.32. & SG A2

Test Package PWS para 1.3.2.4., 4.4.62., & SG A8

Courseware Delivery Schedule

NLT 30 days prior to performance start;

NLT 30 prior to start of option year

Annual COR/SME PWS para 1.3.2.6. & SG A1

Distance Learning Architecture (DLA) and all components

NLT end of phase-in One time SME PWS para 1.3.3.

Student Handbook As approved by the Government in Contractor’s CDS As Required SME PWS para 1.3.5.4.10. &

SG A16

Training Task List As approved by the Government in Contractor’s CDS As Required SME PWS para 1.4.3. & SG

A12 ICW/CBT Instructional Media Package

As approved by the Government in Contractor’s CDS As Required SME PWS para 1.4.3., 4.4.27.

& SG A4

Instructor Led Lesson Package

As approved by the Government in Contractor’s CDS As Required SME PWS para 1.4.3., 4.4.32.

& SG A5

Conference/Meeting Minutes/Agendas

NLT 3 business days following end of conference/meeting As Required Attendees

& COR

PWS para 1.4.4.5 & SG A10

Phase-in/Phase-out Plan Submitted with proposal, and at beginning of Phase-In period Twice COR PWS para 1.4.6.

Contractor’s Progress Status and Management Report

Monthly COR/SME PWS para 1.4.7.1. & SG A9

Quality Control Plan

Submitted with proposal; again after award, and updated as required/directed by the Government

Twice;

As Required COR/SME PWS para 1.4.9.

Training and Certification Plan

Submitted with proposal; again after award Twice COR/SME PWS para 1.5.1

Training Workload Management Plan Submitted with proposal Once COR/SME PWS para 1.6.

Criterion Referenced Objective (CRO) Document PWS para 4.4.16. & SG

A15 Positional Gradesheet SG A17

FOR OFFICIAL USE ONLY (when filled in)

3.2.2 Services Summary. The Services Summary (SS) is a list of critical tasks that must be performed by the Contractor, the performance measure for those tasks, and the method of surveillance the COR/SME will use to validate/inspect these tasks. These items must be assessed/evaluated with results documented in the COR folder and the AMS Monthly COR Report. The table below (Table 2) identifies the Service Summary for the contract and the associated method of surveillance for each performance objective.

Table 2 – Services Summary Performance Objective PWS Ref Performance Threshold Surveillance Method

(1) Contractor shall submit required non-courseware deliverables

Para 1.3.2.2., 1.3.2.4., 1.3.2.6., 1.4.1., 1.4.4.5., 1.4.6., 1.4.7.,

1.5.1. & Appendix 3

Contractor delivers 100% of reports, schedules, plans, and summaries as specified in this PWS.

100% Inspection

(2) Contractor meets all security requirements

Para 1.3.5.2.23., 1.3.5.2.25, 1.3.5.3.9., 1.3.5.5.2.3., 4.1.2., 4.1.4, 4.1.4.1., 4.1.4.2., 4.1.4.3., 4.1.4.4., 4.1.4.5. &4.1.4.12

Contractor ensures all employees adhere to 100% of the security requirements addressed in the PWS.

100% Inspection

(3) Contractor shall develop materials in response to out-of-cycle changes, Urgent Action Directives or major equipment/software changes

Para 1.3.2.10., 1.3.2.11., 1.3.2.12., &Appendix 1 &2

Contractor responds to 100% of all requests for changes in materials.

100% Inspection

(4) Contractor maintains qualified personnel

Para 1.5.1., 1.5.2., 4.1.2., 4.1.3., & 4.2.

Contractor effectively trains and certifies its instructors courseware personnel, and staff support personnel on time IAW the appropriate procedures and ensures that all meet the standard qualifications addressed in the PWS.

Periodic Audit

(5) Contractor performs instructional tasks

Para 1.3.1., 1.3.1.1., 1.3.1.2., 1.3.1.3., 1.3.5.3.11., & Appendix 1

Contractor provides training to 100% of all required personnel within the timeframes specified by accomplishing 100% of the training sessions scheduled.

Contractor exercises flexibility in scheduling to provide maximum training opportunities

Periodic Audit

(6) C2 Staff Support Personnel shall perform tasks.

Para 1.3.5. & all sub-para;

4.1.1., & Appendix 1

The contractor shall perform tasks within allotted time. Periodic Audit

(7) Maintain Distance Learning Architecture

Para 1.3.3. & all sub-para, 3.6.2., & 4.1.1.1.

The contractor shall develop, maintain and update a Distance Learning Architecture capable of delivering all required functions.

100% Inspection

(8) Contractor develops and delivers courseware

Para 1.3.2. & all sub-para;

Appendix 1

Contractor develops courseware IAW its ISD Management Plan, and is delivered IAW the Government approved timelines and the approved Courseware Delivery Schedule.

100% Inspection

(9) Contractor maintains master files Para 1.3.3.6.3., & 1.4.3. Contractor ensures all master files are up to date and accurate within the time frames specified

Periodic Audit

C2 CAT/CWD Solicitation # FA4890-19-R-A025

3.2.3 Service Summary Performance Objectives Checklist. The COR is responsible for developing/updating standard procedures for surveillance checklists and may request outside assistance. Changes to the surveillance checklists (modifying, adding or removing) require the QASP be reviewed in its entirety by the offices identified on the title page. The SS Performance Objective Checklists serve as the guideline for all inspections (see Attachment 4).

3.2.4 Contractor’s Quality P Plan (QCP). A Contractor's internal actions, in the form of a QCP, are written to satisfy contract requirements for written procedures or to provide direction on how Contractor employees are to perform PWS tasks. Any deviations will be treated as deficiencies/non-conformances and handled IAW paragraph 4.1. The QAPC, PM, and COR will review the QCP IAW requirements of the solicitation and subsequent awarded contract. The COR will forward any deficiencies/non-conformances to the QAPC and CO/CM for appropriate action.

3.2.5 SME Assessment. SMEs will have routine interaction with the contractor during the performance of PWS tasks. When a SME identifies technical or funcational inaccuracies, they will notify the contract employee that provided the material or service in error and the C-COR immediately but NLT five (5) business days of the discovery. SMEs will review, and be familiar with, self-inspection procedures in the Contractor’s QCP. The SME is responsible for providing monthly, quarterly, and annual documentation to the C-COR for consolidation and submission in one report to the CO and PM.

3.2.6 Assessment. The normal service summary performance assessment period for CORs is quarterly, by contract year, however, the SME will ensure all service summary performance items are assessed at least quarterly. For this contract, the quarters are: Jul – Sep, Oct – Dec, Jan – Mar and Apr – Jun. If necessary, the COR may request reporting on a monthly basis. All functional areas within the PWS will be inspected at least once per year and will include regular reviews of the contractor's Quality Control Plan (QCP) to ensure contractor's compliance with the QCP.

3.2.6.1 All locations will complete checklists 1, 2, and 4 at least once per contract year.

3.2.6.2 In addition to the three checklists listed above, location-specific checklists to be completed are as follows:

Unit Checklists SME

607ACS 3, 5, 6, 7 Training SMEs

606 ACS 5, 6, 7 Operations SMEs

726 ACS/729 ACS 6, 7 Operations SMEs

EADS/WADS 3, 5, 6, 7 Training/Operations SMEs

ACC TRSS Det 9 and 1 AF/AFNORTH/A3T 3, 7, 8, 9 Courseware SMEs

176 ADS/169 ADS 5, 6, 7 Training SMEs

PACAF A3/6C 3, 6, 9 Operations SMEs

552 ACG 3, 6 Operations SMEs

All TACP Units 6 Operations SMEs

3.3 Deadline to Submit Report.

3.3.1 SS Performance - Submit the SS (Table 2 and Attachment 3 - Quarterly Evaluation Report of Contractor Performance) to the C-COR within five (5) business days after the end of the calendar month. All items in the SS do not need to be reviewed each month, but all items will be reviewed at least once, quarterly. Each COR will forward the completed SS to the C-COR. Reports may be submitted electronically.

3.3.2 Contract Discrepancy – Any contract discrepancy discovered by a COR or SME must be brought to the C-COR’s attention by email within five (5) business days. See Table 3 for types of discrepancy and paragraph 4.1 for non-conformance reporting.

3.3.3 The COR is authorized to perform inspection/surveillance at any time during contract performance IAW FAR 52.212-4 and 52.246-4. CORs will assess contractor SS performance on a monthly basis. All functional areas within the PWS will be inspected by the COR at least once per year and will include regular reviews of the contractor's QCP to ensure contractor's compliance with it, as required by FAR 52.246-11.

3.4 Additional Inspections.

3.4.1 Inspect Contractor’s Quality Control Plan (QCP). A Contractor's internal actions, in the form of a QCP, are written procedures for inspections, records, and security procedures or to provide direction on how Contractor employees are to perform. The COR will review the QCP IAW requirements of the solicitation and subsequent awarded contract, validating the contractor is keeping, and making available to the Government, complete records of its inspection work. Any deviations will be treated as deficiencies/non-conformances and handled IAW paragraph 4.1 The Contractor's QCP is contained in the Instructional Systems Development (ISD) Management Plan.

3.4.2 Evaluate Combating Trafficking In Persons (CTIP). The COR and SMEs will monitor the Contractor’s performance regarding trafficking in persons such that non-compliance with FAR Clause 52.222-50, Combating Trafficking in Persons, is brought to the immediate attention of the CO.

3.4.2.1 Violations and remedies.

3.4.2.1.1 If the contracting officer receives information via the Monthly COR Report or other method indicating that the contractor, a contractor employee, a subcontractor, or a subcontractor employee has failed to comply with the requirements of the clause at FAR 52- 222-50, the contracting officer shall,

3.4.2.1.2 Through the contracting officer’s local commander or other designated representative, immediately notify the Combatant Commander responsible for the geographical area in which the incident has occurred; and

3.4.2.1.3 Information for any investigation and enforcement may be made to:

DoD Inspector General Hotline at http://www.dodig.mil/hotline or 800-424-9098, CTIP Program Manager on-line at http://ctip.defense.gov or by email to ctipreports@osd.pentagon.mil, National CTIP Hotline at 1-888-373-7888, or, CTIP Hotline website at http://www.polarisproject.org.

3.4.2.1.4 Issue a non-conformance.

http://www.dodig.mil/hotline http://ctip.defense.gov/ mailto:ctipreports@osd.pentagon.mil http://www.polarisproject.org/

3.4.3 Perform an IPA. The MFT will conduct a review of Contractor’s performance NET 30 days after the contract start date. SMEs and CORs will submit IPA results to the C-COR for consolidation.

The C-COR will forward the IPA (Attachment 2) to the CM and QAPC.

4.0 CONTRACTOR PERFORMANCE REPORTING

4.1 Non-conformances

Audit or Inspection results that fall below contractual standards shall be identified as non-conformances.

4.1.1 Types of Non-conformance.

4.1.1.1 Minor Non-conformance. A minor non-conformance is a non-conformance, which by itself does not adversely impact mission, safety of personnel and/or equipment, performance (quality), schedule (delivery), or cost. A minor non-conformance normally does not increase risk to the Government.

4.1.1.2 Major Non-conformance. A major non-conformance in this QASP meets the FAR

46.101 definition of both a “Critical nonconformance” and a “Major nonconformance.” It is a non-conformance that adversely impacts, up to preventing performance of, a vital agency mission, safety of personnel and/or equipment, performance (quality), schedule (delivery), or cost. A major non-conformance also increases risk to the Government. An example of increasing risk would be significant number of recurring non-conformances, which indicates inadequate preventive measures or actions, thus lowering the Government’s confidence that quality services will be provided on time and at cost.

4.1.1.3 Non-conformance Notification. If a deficiency is found during inspection of the Contractor’s performance, the C-COR will determine the type of notification based on the category of the non-conformance (see Table 3). Non-conformances pre-identified by the Contractor through the Contractor’s QCP/internal audit does not require a govnerment non-conformance notice; however, if the Contractor fails to take corrective actions in a reasonable time-frame, the C-COR will notify the PM, CO and QAPC for further action.

Table 3 – Non-conformance notifications

Non-conformance Category Notification Type Minor

Step 1: First Notice Step 2: Second Notice

Major Corrective Action Request (CAR)

4.1.1.4 First or Second Notices. As shown in Table 3, First and Second Notices provide written notification from the Government to the Contractor of minor non-conformances. Initial minor non-conformances should be addressed with a First Notice to the applicable Contractor representative. The C-COR will notify the FSM, CO, and QAPC (ACCAMICQAPC@us.af.mil) of the issue, and prepare the notice for CO signature.

4.1.1.4.1 The C-COR will coordinate with the QAPC on the first or second notice NLT five

(5) business days after non-conformance notification, ensuring the first or second notice (Attachment 1) is filled out correctly. The QAPC will provide an electronic copy of the coordinated first or second notice to the CO NLT 10 duty days after discovery of the event. If deemed appropriate, the CO will issue the First or Second notice to the contractor with a mailto:ACCAMICQAPC@us.af.mil suspense date for the contractors reply, sending a courtesy copy email to both the QAPC and

C-COR.

4.1.1.4.2 The contractor shall send the completed first or second notice to the CO and QAPC (ACCAMICQAPC@us.af.mil). If the Contractor fails to respond by the specified due date, the QAPC will notify the CO for action and notify the C-COR to annotate the Contractor’s non-responsiveness in the AMS monthly services summary report. If the Contractor fails to resolve the issue as documented in the Notice, the CO may elevate the issue to the next level (Second Notice or CAR). Corrective/preventive actions initiated and/or completed by the Contractor will be verified by the C-COR on scheduled subsequent inspection and documented accordingly.

4.1.1.5 Corrective Action Requests (CAR) (see Attachment 1). As shown in Table 3, a CAR provides written notification from the Government to the Contractor of significant performance discrepancies. The C-COR will recommend to the FSM, CO, and QAPC that a CAR be issued when previous notifications failed to result in corrective actions or when severe deficiencies exist.

If a CAR is deemed necessary, the QAPC will assist the C-COR in filling out the CAR. The draft CAR will be routed NLT 10 duty days after the event to the CO for review and signature, as the CO is the only person who has authority to issue a CAR to the Contractor.

4.1.1.5.1 To satisfactorily close-out a CAR, the following criteria must be met:

- The Contractor must meet the suspense, and

- The corrective action must have already begun, and

- The Contractor's actions must correct the deficiency, and

- If applicable, be within specified standards.

4.1.1.5.2 If the corrective action is deemed UNSATISFACTORY by the CO, QAPC and C- COR, annotate the CAR accordingly and forward it to the CO with a brief explanation, including an estimate of the Government's loss due to the deficient performance. The estimate may account for loss or destruction of property, supplies, or equipment, and hours lost to re-perform the task. If the task cannot be re-performed, indicate the number of hours it ordinarily would have taken to do the task.

4.1.1.6 The C-COR will maintain a non-conformance log documenting all First Notices, Second Notices and CARs. The log will include the date sent to the Contractor, issue, due date for response and actual response date. If a detailed response is required, include a link to the response and include the response in the applicable COR folder tab.

4.2 COR Monthly Report

The C-COR will complete HQ ACC AMIC’s Acquisition Management System (AMS) Monthly COR Report.

4.2.1 Registration. The C-COR will access AMS via https://cms.acc.af.mil/ and request an account using the following:

4.2.1.1 Organization. Click on link and search for the appropriate organization. If the organization is not in the database, create a new entry by typing in the organization.

4.2.1.2 First Name

4.2.1.3 Last Name

mailto:ACCAMICQAPC@us.af.mil https://cms.acc.af.mil/

4.2.1.4 Email address. Only use a Government-provided email address that is routinely accessed.

4.2.1.5 User Role. Check the box next to Contracting Officer Representative

4.2.1.6 Justification. Enter, “Primary COR on contract FA4890-20-C-#### (C2 CAT/CWD).”

4.2.1.7 Click “Submit Information”

4.2.2 The C-COR certifies the essential services accomplished by the contractor and documents any issues or non-conformances that were identified during the month.

4.2.3 AMS will generate monthly emails to the COR(s) designated on the contract, monthly reports will be completed NLT the 10th calendar day of each month.

4.2.4 A COR will click on the embedded hyperlink in the email, or copy and paste the link onto their browser, OR log into AMS, click on My Requirements, then click on appropriate contract; click on Monthly COR Report.

4.2.5 The monthly report will display questions for the previous month (e.g., answering questions in the month of March will display the monthly report for February).

4.2.5.1 Click on Answer Questions

4.2.5.2 Click on meets, above, below standards or N/A (see Table 4) and provide justification for each answer.

4.2.5.3 After completing the Monthly COR Report, click submit.

4.2.5.4 AMS will only allow one submitted report per month.

Table 4 – Monthly Performance Rating Definitions

Rating Definition Above Standards

Contractor performance exceeds contractual requirements and is beneficial to the government. To justify exceeds standards, state which PWS or SOW requirement(s) was exceeded (to include the paragraph number) along with how exceeding this requirement was a benefit to the government.

Meet Standards

Contractor performance meets contractual requirements. To justify meets standards, specify the PWS paragraph/requirement and state that the contractor met the requirement.

Below Standards

Contractor performance does NOT meet some contractual requirements. To justify below standard, identify an event(s) that the Contractor had trouble overcoming and state how it impacted the Government. This rating should be supported by referencing any non-conformance notification sent to the Contractor. Under Corrective Action, state what correction action the contractor has taken or is planning to take based on the contractor’s response to the non-conformation notification.

N/A Contractor was not contractually required to perform specific task or specific question is not applicable for evaluated period.

4.3 Annual Contract Performance Assessment Reporting System (CPARS)

An annual CPAR will be written on all Services contracts that exceed total contract value dollar reporting thresholds. The CPARS website, https://www.cpars.gov/, is the Federal Government’s Enterprise Solution for collection of Contractor Past Performance Information (PPI), as required by the FAR. CPARS is a web-enabled application that collects and manages a library of automated Contractor report cards. The C-COR and PM (if different individuals) will be designated as Assessing Official Representatives (AOR) in CPARS. The AOR will use objective evidence from Contractor deliverables, Monthly/Quarterly Reports, 1st/2nd notices, CARs, WAWF invoices and any other validated information provided by CORs or the customer, that could be used for the CPAR assessment. The AOR will send the Draft CPAR to the Assessing Official (AO) within 20 calendar days following the end of the Period of Performance (PoP). The table below (Table 5) is an excerpt from the AMIC CPARS Roles and Responsibilities Policy Letter.

Table 5 – Performance Assessment Report Routine Timelines*

CPAR Role: Timeframe Action in CPARS AOR +20 calendar days following the end of the Period of Performance (PoP)

Draft (Narrative and rating) report and send to CO

AO +40 calendar days following the end of the PoP

Validate narrative/ratingcontract information and send to Contractor Representative

Contractor Representative 60 calendar days Comment and return to CO AO (Contractor Concurs or does not respond)

7 calendar days but not to exceed PoP end +107 days

Completes CPAR

AO (Contractor Non-Concur) RO (Contractor Non-Concur

14 calendar days but not to exceed PoP end +114 days

CO forwards to RO; RO reviews, updates and completes CPARS

*Focal Point will notify AOR, AO, RO or division chief for all exceeded time frames

4.4 Wide Area Workflow (WAWF).

The C-COR will accept or reject all Receiving Reports submitted by the Contactor in WAWF within 7 days of submittal by the Contractor. To access the receiving report, log into WAWF, select the Government, Acceptor, Acceptance Folder tab; then type in the contract and delivery order (contract/task order) number. The Shipment Number column in WAWF is the Receiving Report number.

4.4.1 Validating Receiving Reports. The C-COR will review and validate all WAWF receiving reports (Shipment Number in WAWF) ensuring that:

• The Contractor submitted a combination invoice/receiving report

• The report reflects the correct contract and TO number

• Beginning and ending dates being billed are stated and accurate

• The correct CLIN(s)/SLIN(s) is(are) charged for billing period

• A description of the services is provided (Firm Fixed Price, Labor Hour, Travel, Materials)

• The correct amount is charged, to include applicable handling rates

• Invoiced travel was pre-approved https://www.cpars.gov/

• Back-up documentation is attached that includes a Contractor invoice with current and cumulative hours (for FFP LOE contracts) and amounts, as applicable

• A breakout of any other charges, as agreed to by the CO or required by the Task Order, is attached in WAWF, or has been provided to the C-COR

4.4.2 Certifying and Accepting Receiving Reports. Inspection of Services is ALWAYS performed by the appointed COR. The C-COR will validate and certify services through non-conformance notifications (para.4.1) and COR monthly reports (para.4.2.2.). After validation of services, the C- COR will select the “CQA” (Contract Quality Assurance) button to verify that the invoice was accurate, this step will normally select the Acceptance button, also. If anyone other than the C-COR is accepting services for the contract, the C-COR MUST certify in writing to the acceptor that services have been inspected/validated, the acceptor must then state in the comments block (under Misc Info tab) that the C-COR has reviewed, validated and certified services/invoices. In all cases, the C-COR will retain documentation (electronic or hard copy) showing COR validation and certification, as well as final acceptance of Contractor invoices.

4.4.3 Expenditure Log. After certification of an invoice in WAWF, the C-COR will update the expenditure log documenting/tracking certified invoice expenditures against funds allocated on the contract against each CLIN. Funding belongs to the organization where the Requirement Owner resides; the expenditure log allows the C-COR to ensure that funds are available for the work performed, to include travel and materials.

4.4.4 Rejecting Receiving Reports. If a receiving report is rejected in WAWF, the C-COR will go to the Miscellaneous Info tab to complete the comments block as follows:

4.4.4.1 List details regarding all unsatisfactory services or incorrect invoice information, and

4.4.4.2 Instruct the Contractor to contact DFAS to have the invoice rejected back to the Contractor; this will allow the Contractor to make the same correction on both the receiving report and invoice, since combination invoices/receiving reports are required.

4.4.4.3 The rejecter will then return to the Header tab and click the “Reject to Initiator” block.

After rejecting a receiving report for unsatisfactory services, e-mail the CM/CO the contract number, receiving report number, and reason for rejection. Rejections for incorrect invoice information will be tracked by the COR.

5.0 COR FOLDER.

The COR will establish and maintain a COR folder for each contract/task order; an electronic version of the folder may be kept, however, it must be maintained in an area where alternate CORs and COR Management will have access to it. The COR folder must contain the tabs and contents outlined below in Table 6. All surveillance records are considered FOR OFFICIAL USE ONLY documents, and must be marked and safeguarded as such. Documentation is kept for the life of the contract and provides necessary continuity should COR duties transition during the course of the contract. COR Management and the COR will receive instructions from the CO on disposition or archival procedures upon contract completion.

5.1 SME Files

5.1.1 A centralized master file will be established at each location for SME reference and containing all the items listed in Table 7. Individual SMEs will organize their files in such a manner to reference the master file or effectively locate essential information when needed.

5.1.2 File Disposition.

The COR will receive instructions from the CO on disposition or archival procedures for any SME records upon contract completion. SME inspection reports posted to the TRSS Sharepoint will be kept on the site for one year. Permanent storage for all SME records will be at the place of performance. The COR will notify the SME’s prior to deleting any files more than one year old to ensure SMEs have properly archived their own records.

Table 6 – COR Folder Tabs & Contents

Tab Title Items to Include 1 General Correspondence If electronic, must be available upon request and for the life of the contract 2 COR/SME designation Letter(s) Assignment/nomination letter(s), designation letter(s), copy of termination letters, and Government/Contractor relationships guidance

3 COR/SME Training COR/SME certificates for DAU, Ethics, CTIP, and CO/QAPC or CO Led training

4 Contract/Task Order, Modifications, PWS, Pricing

Contract for current level of effort, any modifications and most current PWS/amendment on contract, and Contractor’s contract/task order pricing

5 Quality Assurance Surveillance Plan, Surveillance Checklist

Contract/task order QASP, approved surveillance checklist, and completed surveillance checklists

6 All Required Contractor Plans Strike Mitigation Plan ISD Management Plan Phase-In/Phase-Out Plan Quality Control Plan Training and Certification Plan Training Workload Management Plan

7 First & Second Notices, Corrective Action Reports (CAR)

Non-conformance log, First and Second Notices and CARs issued to Contractor

8 Contractor Monthly Reports Contractor’s Progress Status and Management Report 9 WAWF Invoices WAWF invoices can be maintained within WAWF, however, WAWF attachments must be inspected and maintained in Tab 9.

10 COR Expenditure Log Updated with latest invoice and funding balance 11 DD Form 254 If applicable 12 Monthly Log The COR will maintain a written or electronic log which can be used to assist in tracking non-conformances and any other information the COR deems necessary or appropriate.

13 CPARS All documentation used to develop a CPAR

Table 7 – SME File Tabs & Contents

Tab Title Items to Include 1 General Correspondence If electronic, must be available upon request and for the life of the contract

SME Documentation

(a) Nomination Letters,

(b) Relief From Duties Letters,

(c) Duties and Responsibilities Letters

3 Quality Assurance Surveillance Plan Contract/task order QASP 4 SME Training Material and

Guidance As provided by the PM/C-COR/COR

Contract

Conformed copy of awarded contract, including all modifications, SF1449, DD254, PWS, and PWS addendums, as distributed by the COR

Contractor Plans

ISD Management Plan Quality Control Plan Training and Certification Plan Strike Mitigation Plan (Luke AFB, only)

Contractor Certifications

The certification and qualification letters of each contract instructor (IAW procedures in Contractor’s Training and Certification Plan)

Surveillance Documents

Contract Monitoring and Surveillance Reports, SS Performance Objective Checklists, SME Reports, and any locally developed SME surveillance documents

9 First & Second Notices, Corrective Action Reports (CAR)

Non-conformance log, First and Second Notices and CARs issued to Contractor

Contractor Reports

Contractor’s Monthly Progress Status and Management Report, and, any other contractor-provided reports

11 Style Guide Most current version

Attachment 1 – Non-Conformance Report (Example Only)

QMSF 8.3-1

Non-Conformance F

Attachment 2 – Initial Performance Assessment (IPA) (Example Only)

MEMORANDUM FOR ACC AMIC/PKB Date

FROM: (COR’s Office)

SUBJECT: Initial Performance Assessment for Command and Control Contract Aircrew Training, Courseware Development, and Distance Learning Services on Contract FA4890-20-C-####

Contract Dollar Value: $

1. Per the contract Quality Assurance Surveillance Plan, an initial assessment of Contractor performance is due no earlier than 30 days after the Contractor assumes full performance responsibility (i.e., after completion of transition).