Draft_552_ACW_E-3_CAT_CWD_QASP_14_Mar_2019.pdf

Quality Assurance Surveillance Plan Contract Monitoring and Surveillance Report

BY ORDER OF THE DIRECTOR HQ ACC ACQUISITION

MANAGEMENT AND

INTEGRATION CENTER

XX XXX 2018

552 Air Control Wing Aircrew

Training and Courseware Development (FA4890-20-C-00XX)

Pages: 40

COMPLIANCE WITH THIS DOCUMENT IS MANDATORY

NOTICE: This Document is available in digital format through ACC AMIC/DRQQ

OPR: ACC TRSS/ACQ Reviewed by: ACC TRSS/ACQ Druie D. Barnard, GS-13 Raymond L. Churchill, GS-14 Contracting Officer Representative Commander, ACC Training Support Sqd.

Reviewed by: ACC AMIC/DRQQ Accepted by: ACC AMIC/PKBB Patrick J. Driscoll, NH-03 Tara M. Holloway, NH-03 Quality Assurance Program Coordinator Contracting Officer

TABLE OF CONTENTS

SECTION

1.0 SCOPE

1.1 Authority

1.2 Purpose

1.3 Related Documents

2.0 MANAGEMENT LOCATIONS AND KEY PERSONNEL

2.1. Location

2.2. Contracting Management Team

2.3. Contractor Key Personnel

3.0 CONTRACT SURVEILLANCE

3.1 Surveillance Methods

3.2 Surveillance Documents

3.3 SME responsibilities:

3.4 COR Inspections

4.0 CONTRACTOR PERFORMANCE REPORTING

4.1 Non-conformances

4.2 COR Monthly Report

4.3 Annual Contract Performance Assessment Reporting System (CPARS)

4.4 iRAPT (Invoice, Receipt, Acceptance and Property Transfer) formerly Wide Area

Work Flow (WAWF)

5.0 COR FOLDER

Attachment 1 - Contract Monitoring and Surveillance Report Attachment 2 - Non-Conformance Template (Example Only) Attachment 3 – Sample SME Nomination Letter Attachment 4 – Sample SME Relief from Duties letter Attachment 5 – Sample Quarterly Surveillance Report Attachment 6 – Individual Performance Objective Inspection Checklists Attachment 7 – Guidelines For Courseware Deliverables Evaluations………………………….43

TABLE

Table 1 - Deliverables Table 2 - Services Summary Table 3 - Non-conformance notifications Table 4- Monthly Performance Rating Definitions Table 5- Performance Assessment Report Routine Timelines Table 6 -COR Folder Tabs & Contents

2.1.0 SCOPE

The Quality Assurance Surveillance Plan (QASP) is applicable to personnel performing contract surveillance inspections in accordance with Federal Acquisition Regulation (FAR) Part 46, Quality Assurance; Department of Defense Instruction (DODI) 5000.72 DoD Standard for Contracting Officer’s Representative (COR) Certification; DOD FAR Supplement (DFARS) Subpart and PGI 201.602-2 , Contracting Officer Responsibilities; Air Force FAR Supplement (AFFARS) Mandatory Procedures (MP) 5301.602-2(d) Designation, Assignment, and Responsibilities of a Contracting Officer’s Representative (COR); Air Force Instruction (AFI) 63-501, Air Force Acquisition Quality Program; Air Force Instruction (AFI) 63-138, Acquistion of Services; and organizational policy. It is designed to provide the Contracting Officer Representative (COR) a systematic surveillance method for each service identified in the Performance Work Statement (PWS). The Functional Service Manager and COR develop the methods for administering and evaluating performance. The Quality Assurance Program Coordinator (QAPC) and Contracting Officer (CO) or Contract Manager (CM), will review these methods, which are then accepted or rejected by the Contracting Officer (CO).

2.1.1.1 Authority

The following FAR clauses provide authority for this QASP:

Applicable inspection clauses:

52.246-4, Inspection of Services -- Fixed-Price

Commercial Items (FAR Part 12) 52.212-4, Contract Terms and Conditions -- Commercial Items (acceptance after delivery)

FAR 46.401, Government Contract Quality Assurance. This clause directs that Government contract quality assurance shall be performed as necessary to determine that the supplies or services conform to contract requirements. Quality assurance surveillance plans will be prepared, and should specify the work requiring surveillance and the method of surveillance.

FAR 46.407 -- Nonconforming Supplies or Services – This clause specifies the contracting officer rejects supplies or services not conforming in all respects to contract requirements. The contracting officer ordinarily must give the contractor an opportunity to correct or replace nonconforming supplies or services when this can be accomplished within the required delivery schedule; correction or replacement must be without additional cost to the Government.

46.501 – Acceptance (General) Acceptance constitutes acknowledgment that the supplies or services conform with applicable contract quality and quantity requirements, except as provided in this subpart and subject to other terms and conditions of the contract. Acceptance may take place before delivery, at the time of delivery, or after delivery, depending on the provisions of the terms and conditions of the contract. Supplies or services shall ordinarily not be accepted before completion of Government contract quality assurance actions. Acceptance shall ordinarily be evidenced by execution of an acceptance certificate on an inspection or receiving report form or commercial shipping document/packing list.

Formatted: Outline numbered + Level: 1 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0", Tab stops: 0.38", List tab

Formatted: Outline numbered + Level: 2 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0"

FAR 52.246-4(a), Commercial Items Inspection/Acceptance.- This clause provides the Government the right to inspect or test any supplies or services that have been tendered for acceptance. In addition, the Contractor shall provide and maintain an inspection system acceptable to the Government. If the Contractor fails to take the necessary action to correct any services that do not meet contract requirements, the Government may perform the services and charge the Contractor for any direct cost incurred by the Government.

2.2.1.2 Purpose

This QASP provides a systematic method to assess and evaluate the services received from the Contractor, however, this QASP is not considered part of the contract. It defines the policy, procedures, and associated methods used by Multi-Functional Team (MFT) for planning, preparing, performing, analyzing, and reporting Contractor performance. In the event this QASP conflicts with the contract or task order, the contract or task order will take precedence.

2.3.1.3 Related Documents

2.3.1.1.3.1 CAF Cockpit/Crew Resource Management Contract number FA4890-18-C-0003.

2.3.2.1.3.2 PWS solicition number FA4890-XX-R-00XX.

3.2.0 MANAGEMENT LOCATIONS AND KEY PERSONNEL

a.2.1. Location

Program and Contract Management COR Management

ACC AMIC ACC TRSS/ACQ

129 Andrews Blvd (Bldg 664) 130 Andrews St., STE 118 Joint Base Langley-Eustis VA 23665 Joint Base Langley-Eustis VA 23665 b.2.2.Contracting Management Team i.2.2.1. Contracting Officer (CO). CO duties are detailed in FAR subpart 1.602-2, Contracting

Officer Responsibilities, and AFFARS MP5346.103, Contracting Officer Responsibilities.The CO is the only Government agent authorized to award or modify contracts; therefore, the CO is the only person authorized to contractually obligate the Government. The CO is also the individual who will sign any notifications to Contractors regarding performance issues and accepts Government surveillance methods utilized for each TO. The CO approves/appoints all CORs assigned by the Functional Service Manager.

NOTE: The CO can be reached at DSN: 574-3222 or Commercial 757-764-3222.

ii.2.2.2. Contract Manager (CM) The CM is the Government agent within the contracting office who performs day-to-day administration of the contract. The CM may also be the CO if the person has a warrant.

NOTE: The CM can be reached at DSN: 574-3222 or Commercial 757-764-3222.

Formatted: Outline numbered + Level: 2 + Numbering

0" + Indent at: 0"

Formatted: Outline numbered + Level: 2 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0"

Formatted: Outline numbered + Level: 1 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0", Tab stops: 0.38", List tab

Formatted: Indent: Left: 0", Hanging: 0.5", Outline numbered + Level: 2 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at: 0" + Indent at: 0.25"

Formatted: Indent: Left: 0", Hanging: 0.5", Outline numbered + Level: 2 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at: 0" + Indent at: 0.25"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5"

Formatted: Outline numbered + Level: 3 + Numbering

0" + Indent at: 0.5" iii.2.2.3. Functional Service Manager (FSM). Executes management and oversight for the delivery of contractually acquired services. Keeps up-to-date on mission changes that may drive the need for a contract modification. Ensures Contract Performance Assessment Reports (CPARS) are conducted. Reviews and approves individual’s nomination as CORs to the CO to enable appropriate contract surveillance. Reviews the COR’s contract performance documentation on a regular basis to ensure performance is compatible with contract and A3 mission objectives.

NOTE: The FSM for this contract is the ACC TRSS/CC. You can reach the TRSS/CC at DSN: 574-8600 or Commercial 757-764-8600.

iv.2.2.4. Requirement Owner/COR Supervisor. This role may be performed by the FSM.

Ensures continuous contract monitoring by a qualified COR. Review, approve, and submit COR nominations packages to the CO. Ensures personnel are properly trained and technically qualified to perform COR duties, have no personal conflict of interest, and COR responsibilities are assessed on performance report. Evaluate the individual’s performance as a COR and solicit feedback from the CO to include in the overall evaluation.

Evaluation of CORs performance occurs if the COR is a dedicated full-time, part time or if the CORs duties are assigned as additional responsibilities. Ensures the COR has the necessary clearance for the contract. Ensures the COR is provided and completes all required annual and three year refresher training along with any additional training mandated by the contracting officer. Receives contract specific training by the QAPC.

v.2.2.5.Quality Assurance Program Coordinator (QAPC). The QAPC integrates the quality contract requirements into the quality assurance program and acts as the liason between the contracting and functional organizations; specifically reviewing the PWS and QASP to ensure that requirements are clearly stated, enforceable, and measureable. Trains CORs and COR Management (e.g. COR Supervisor, FSM) on the contracting requirements associated with the quality assurance program and MAJCOM/DRU procedures. Tracks all contract nonconformances, assisting CORs in drafting nonconformances notices (1st/2nd notice/CAR) NOTE: The QAPC can be reached at DSN: 574-9372 or Commercial 757-764-9372

1.2.2.6 Contracting Officer Representative (COR). The COR is ultimately responsible for implementing all QA matters and procedures. COR(s) should be at a level where other duties don’t prevent proper oversight of Contractor performance. The COR must maintain both technical competency and evaluation proficiency in Contractor surveillance procedures. The COR must also have enough knowledge of Contractor activities to properly review and disposition monthly invoices. The COR cannot assume duties until assigned by the requiring agency, completed COR required training (CO/QAPC and CO led training), and appointed by the CO. The COR is responsible for overall surveillance to include surveillance conducted by functional SMEs. If functional SMEs are utilized, the COR will develop and implement a plan to collect SME surveillance documentation for consolidation into a single surveillance report, submitted quarterly but could be increased to monthly if contract performance is degraded. The COR will also use the TRSS Det monthly report as a resource for obtaining contract information on a monthly basis.

Formatted: Outline numbered + Level: 3 + Numbering

0" + Indent at: 0.5"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 6 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

1.2.2.7.Functional Subject Matter Experts (SME). SME are responsible for validating contractor performance in the SME respective area of expertise and providing documentation (as required in DoDI 5000.72, DFARS PGI 201.602, AFFARS MP 5301.602-2(d) and this instruction) to the COR.

SMEs are nominated by their supervisor then trained and approved by the COR. When no longer required or able to perform SME duties (due to PCS, change of duties, etc.), the SME must be formally relieved of duties by the office which submitted the original nomination, or by the COR). SMEs are technical experts and can only recommend changes to courseware or training whenever a technical error is discovered. Any technical error discovered by a SME should be resolved at the lowest level possible. Any disputes between contractor and SME regarding technical errors will be elevated to the COR immediately or within 5 working days at the most.

SMEs shall receive training conducted by the COR prior to assuming SME duties.

Training will, at a minimum, consist of functional SME responsibilities, ethics, and review of applicable regulations along with a detailed discussion of the contract.

2.2.3.Contractor Key Personnel.

1.2.3.1.Program Manager. The Contractor’s Program Manager is responsible for the overall execution of the contract. This individual will serve as the focal point for the CM/CO and COR assigned to the contract and may interface with the customer, as needed. The PM for this contract is __________________________.

2.2.3.2.Site Manager. The Site Manager is responsible for technical or quality control oversight, and day to day operations of the contract. The SM for this contract is ______________.

2.3.0CONTRACT SURVEILLANCE.

The COR and SME functions are responsible for a wide range of surveillance requirements that effectively measure and evaluate Contractor performance. The Contractor, not the Government, is responsible for contract performance management and Quality Control (QC) actions to meet the terms of the contract. However, because of the non-commercial nature of the work performed by Contractors, the Government cannot rely solely on the Contractor’s quality program but must have a method to oversee performance of the entire contract. The Government does this through the QASP and can inspect or test all services called for by the contract, to the extent practicable, at all times and places, during the term of the contract.

1.3.1Surveillance Methods.

Four methods of surveillance will be used: Initial Contract Performance Review, one hundred percent inspection, periodic audits, and end of course surveys. Other methods of surveillance must be fully documented, reviewed by the QAPC and approved by the CO before implementation. Information obtained from these activities may be used by the FSM/COR/Requirement Owner to ascertain whether or not Contractor performance is compatible with contract and mission objectives and by the CO for actions relating to the Contractor.

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 7 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Indent: Left: 0", Hanging: 0.5", Outline numbered + Level: 2 + Numbering Style: 1, 2, 3, … + Start at: 2 + Alignment: Left + Aligned at: 0" + Indent at: 0.25", Tab stops: Not at 1"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 1 + Numbering Style: 1, 2, 3, … + Start at: 2 + Alignment: Left + Aligned at:

0" + Indent at: 0.33", Tab stops: Not at 0.5"

Formatted: Outline numbered + Level: 2 + Numbering

0" + Indent at: 0.33", Tab stops: Not at 1"

1.3.1.1 Initial Contract Performance Review (ICPR). The initial evaluation of contractor performance is a joint determination by the MFT that may include CO, CM, QAPC, and CORs to determine if the contractor completed transition, successfully started performance, is fully operational, and is within the estimated cost, schedule, and performance parameters of the contract. The review shall take place no later than 30 days after the contractor assumes full performance responsibility. The COR will complete the report received from the CM then return it to the CM.

2.3.1.2 One Hundred Percent Inspection. This method will be used for Contractor submissions, products, and deliverables (tangible items). CORs/SMEs will use the One Hundred Percent Inspection method to evaluate all reports delivered under paragraph 3.2. The COR/SME, with assistance from the CM and CO if necessary, accepts or rejects these products based on the criteria and performance standards as outlined in the Services Summary. The COR/SME also documents all results of inspections actions, and subsequent Contractor re-performance taskings, if applicable.

3.3.1.3 Periodic Audit. This type of surveillance consists of COR/SME evaluating products/services not surveilled via one hundred percent inspection or customer survey.

Periodic audits are implemented to meet COR-defined reporting requirements.

4.3.1.4 End of Course Surveys. The COR and SMEs will use the contractor provided end of course surveys as a surveillance method and improving future training materials.

2.3.2Surveillance Documents

1.3.2.1 Deliverables. All deliverables will be reviewed for timeliness, accuracy and format. If a deliverable due date falls on a weekend or holiday, the Contractor shall submit the deliverable on the last work day prior to the due date.

Table 1 - Deliverables

Deliverables PWS Reference OPR Due Date

Courseware Delivery Schedule (CDS) PWS para 1.6.5 SME

NLT 45 days after contract award no later than 30 calendar days prior to the beginning of each fiscal year

Quality Program Plan PWS para 4.9.4 COR Concurrently with the contractor's proposal.

Academic Scheduling. PWS para 1.12.2 SME Weekly

Conference/Meeting Minutes PWS para 1.13 COR Within 5 business days of the conference/ meeting.

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 2 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.33", Tab stops: Not at 1"

Formatted: Outline numbered + Level: 3 + Numbering

Strike Mitigation Plan PWS para 4.9.5 COR Within ten days of contract award

Training Workload Management Plan PWS para 4.9.1 COR Concurrently with the contractor's proposal.

Instructional Systems Development Management Plan

PWS para 4.9.2 COR/SME Concurrently with the contractor's proposal.

Transition Plan PWS para 4.9.6 COR/SME Concurrently with the contractor's proposal.

Instructor Training and Certification Plan PWS para 4.9.3 COR/SME Concurrently with the contractor's proposal.

Unit Lesson Strategy Report Style Guide SME Deliver as identified in the contractor's approved Courseware Delivery Schedule.

Lesson Overview Style Guide SME Deliver as identified in the contractor's approved Courseware Delivery Schedule.

Instructor Media Package Style Guide SME Deliver as identified in the contractor's approved Courseware Delivery Schedule.

Lesson Package Style Guide SME Deliver as identified in the contractor's approved Courseware Delivery Schedule.

Lesson Plan Style Guide SME Deliver as identified in the contractor's approved Courseware Delivery Schedule.

Lesson Study Guide Style Guide SME Deliver as identified in the contractor's approved Courseware Delivery Schedule.

Test Package Style Guide SME

Deliver as identified in the contractor's approved Courseware Delivery Schedule.of notification of acceptance of the draft.

Training Evaluation Document Style Guide SME

Submitted annually identified in the contractor's approved Courseware Delivery Schedule.

Contractor’s Progress, Status and Management Report

PWS Attachment 1 and Style Guide

SME

Submitted monthly.

E-mailed within the first 5 work days of the month for the previous month.

Course Syllabus Style Guide SME Submitted annually identified in the contractor's approved Courseware Delivery Schedule

Syllabus Review Report Style Guide SME Submitted annually identified in the contractor's approved Courseware Delivery Schedule

Training Task Lists Style Guide SME Submitted annually identified in the contractor's approved Courseware Delivery Schedule.

Courseware Revision Summary Report Style Guide SME

Submitted annually identified in the contractor's approved Courseware Delivery Schedule.

Crtierion Referenced Objective Document Style Guide SME

Submitted annually identified in the contractor's approved Courseware Delivery Schedule.

Student Handbook Style Guide SME Submitted annually identified in the contractor's approved Courseware Delivery Schedule.

Positional Grade Sheet Style Guide SME Submitted annually identified in the contractor's approved Courseware Delivery Schedule.

GFE Inventory PWS Para 3.3.1 SME Submitted at contract start and annually

2.3.2.2 Services Summary. The Services Summary (SS) is a list of critical tasks that must be performed by the Contractor, the performance measure for those tasks, and the method of surveillance the COR/SME will use to validate/inspect these tasks. These items must be assessed/evaluated with results documented in the COR folder and the CMS monthly COR Report. The table below identifies the Service Summary for the contract.

Table 2 - Services Summary

PERFORMANCE

OBJECTIVE

PWS

PARA PERFORMANCE MEASURE METHOD OF

SURVEILLANCE

Para 4.9.3 / 5.4, A5.3, A5.4, A5.5

Contractor shall provide qualified personnel IAW Instructor Training and Certification Plan ( PWS 4.9.3 ) , Contractor Qualifications ( PWS 5.4, A5.3,A5.4, A5.5 ), and current instructions to meet USAF performance standards.

Acceptable Quality Level: 100%

Para 1.7.2, 1.7.3, Appendix C

Contractor shall ensure all academic events are attended by qualified instructors and conducted IAW this PWS 1.7.2 and

1.7.3 . Shall provide a proficient mission crew member for all crew positions IAW current instructions and guidance to meet USAF performance standards.

Acceptable Quality Level: 100%

Para 1.7.4, Appendix C

Contractor shall ensure all training device events are attended by qualified instructors and conducted IAW this PWS 1.7.4 . Shall provide a proficient mission crew member for all crew positions IAW current instructions and guidance to meet USAF performance standards.

Acceptable Quality Level: 100%

Appendix F Para 5.2

Contractor shall ensure flight training events are attended by qualified instructors and conducted IAW this PWS Appendix FA5.2 . Shall provide a proficient crew member for all crew positions required by this PWS and IAW current instructions and guidance to meet USAF performance standards.

Acceptable Quality Level: 100%

Para 1.9 and Appendix D

Contractor develops all courseware IAW with PWS 1.9 and Appendix D and the contractors ISD Management Plan. All courseware (New, Review and Revised) as outlined in the annual Courseware Delivery Schedule will be delivered by 31 January of each year. Contractor will provide approximately one-fourth of the courseware to be developed on a quarterly basis. Deviations to the delivery schedule must be coordinated through the COR.

Acceptable Quality Level: 95%

(6) Contractor developS, maintains, updates, reviews, and delivers MSTP for generation of AWACS simulator events.

Para 1.12.6 Appendix E

Contractor shall ensure compatibility with AWACS aircraft software and version modifications and to allow MSTP operate of all simulators IAW PWS 1.12.6.

Acceptable Quality Level: 100%

The COR will conduct periodic surveillance on a monthly basis.

Para 4.6 Contractor ensures all employees adhere to 100% of the security requirements addressed in the PWS.

(PWS Para 4.6.)

(4) Contractor performs Flight instruction.

The Govt will conduct periodic surveillance on an annual basis via annual and no notice check rides / observations.

(5) Contractor develops and delivers courseware.

The COR will conduct periodic surveillance on a monthly basis.

(7) Contractor meets all security requirements

The COR will conduct periodic surveillance on a quarterly basis.

(1) Contractor maintains qualified personnel.

COR will conduct periodic surveillance on a quarterly basis

(2) Contractor performs academic instruction.

The COR will conduct periodic surveillance on a monthly basis.

(3) Contractor performs Training Device Instruction.

The COR will conduct periodic surveillance on a monthly basis.

3.3.2.3 Service Summary Performance Objectives Checklist. The COR is responsible for developing/updating standard procedures for surveillance checklists and may request SMEs assistance. The SS Performance Objective Checklists serve as the guideline for all inspections (see Attachment 7).

4.3.2.4 Contractor’s Quality Control Plan (QCP). A Contractor's internal actions, in the form of a QCP, are usually written to satisfy contract requirements for written procedures or to provide direction on how Contractor employees are to perform. Any deviations will be treated as deficiencies/non-conformances and handled IAW paragraph 4.1. The COR will review the QCP IAW requirements of the solicitation and subsequent awarded contract. The COR will forward any deficiencies/non-conformances to the QAPC and CO/CM for appropriate action.

1.3.3SME responsibilities:

3.3.1 SMEs are responsible for the day to day technical review of training materials in the form of courseware or classroom training. When a SME discovers training materials that are not technically accurate, they will notify the contract instructor who supplied the materials or taught the class of the error. If the contractor disputes the information, the SME will notify the COR immediately or within 5 working days. The SME and contractor should work together to rectify any technical issues at the lowest level possible. SMEs will review and be familiar with self-inspection procedures in the Contractor’s QCP.

3.3.2 Assessment - The normal service summary performance assessment by the SME is quarterly by contract year. For this contract, the quarters are Feb – Apr, May – Jul, Aug – Oct and Nov – Jan. If necessary, the COR may request reporting on a monthly basis.

3.3.3 Deadline to Submit Report.

1.3.3.1.1 Services Summary (SS) Performance - Submit the SS (Table 2) to the COR within five (5) working days after the end of the reporting period. All items in the SS do not need to be reviewed each quarter but all items will be reviewed at least once annually.

PERFORMANCE

OBJECTIVE

PWS

PARA PERFORMANCE MEASURE METHOD OF

SURVEILLANCE

(8) Contractor maintains master files Para 1.10.3 Contractor ensures all master files are maintained accurately and up to date in accordance with this PWS 1.10.3 .

The COR will conduct periodic surveillance on a quarterly basis.

(9) Contractor submits required data and reports.

Para 1.12.1 Attachment

Contractor submits 100% of all reports and schedules as specified in this PWS 1.12.1 and Attachment 1 .

The COR will conduct periodic surveillance on a monthly basis.

(10) Contractor performs academic schduling and registrar support

Para 1.9.2.2

Contractor shall ensure accuracy, performance, and timely dissemination of the total management system program.

Contractor shall ensure no interruption to scheduled training events for aircrew student qualification training.

Acceptable Quality Level: 100%

The COR will conduct periodic surveillance on a monthly basis.

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 2 + Numbering Style: 1, 2, 3, … + Start at: 3 + Alignment: Left + Aligned at:

0" + Indent at: 0.33", Tab stops: Not at 1"

Formatted: Outline numbered + Level: 4 + Numbering

0" + Indent at: 0.5", Tab stops: Not at 2"

Each SME will forward the completed SS to the COR. Reports may be submitted electronically.

2.3.3.1.2 Contract Discrepancy – Any contract discrepancy discovered by the SME must be brought to the COR’s attention by email within 5 working days. In this case, contract discrepancies are anything other than normal courseware or training inaccuracies that can be corrected immediately.

2.3.4COR Inspections The COR is authorized to perform inspection/surveillance at any time during contract performance IAW FAR 52.212-2 and 52.212-4. CORs will assess contractor Services Summary performance on a monthly basis. All functional areas within the PWS will be inspected by the COR at least once per year and will include regular reviews of the contractor's Quality Control Plan (QCP) to ensure contractor's compliance with the QCP.

1.3.4.1 Inspect Contractor’s Quality Control Plan (QCP). A Contractor's internal actions, in the form of a QCP, are written procedures for inspections, records, and security procedures or to provide direction on how Contractor employees are to perform. The COR will review the QCP IAW requirements of the solicitation and subsequent awarded contract, validating the contractor is keeping, and making available to the COR/SME, complete records of its inspection work. Any deviations will be treated as deficiencies/non-conformances and handled IAW paragraph para 4.1 The contractor's QCP can be contained in the contractor's Instructional Systems Development (ISD) Management Plan or as a separate document.

2.3.4.2 Evaluate Combating Trafficking In Persons (CTIP). The COR/SME will monitor the contractor’s performance regarding trafficking in persons such as that non-compliance with FAR clause 52.222-50, Combating Trafficking in Persons, is brought to the immediate attention of the contracting officer.

a. Violations and remedies.

a.i. If the contracting officer receives information via the Monthly COR Report or other method indicating that the contractor, a contractor employee, a subcontractor, or a subcontractor employee has failed to comply with the requirements of the clause at FAR 52-222-50, the contracting officer shall, b.ii. Through the contracting officer’s local commander or other designated representative, immediately notify the Combatant Commander responsible for the geographical area in which the incident has occurred; and c.iii. Information for any investigation and enforcement may be made to;

d.iv. DoD Inspector General Hotline at Http://www.dodig.mil/hotline or 800-424-9098, Program Manager on-line at http://ctip.defense.gov or by email to ctipreports@osd.pentagon.mil, National CTIP Hotline at 1-888-373-7888 or to the the CTIP Hotline website at http://www.polarisproject.org.

e.v. Issue a non-conformance.

3.4.3 Perform an Initial Contract Performance Review (ICPR). The COR will conduct review of contractor’s performance 30 days after the contract start date.

Formatted: Outline numbered + Level: 4 + Numbering

0" + Indent at: 0.5", Tab stops: Not at 2"

Formatted: Outline numbered + Level: 2 + Numbering Style: 1, 2, 3, … + Start at: 3 + Alignment: Left + Aligned at:

0" + Indent at: 0.33", Tab stops: Not at 1"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Numbered + Level: 1 + Numbering Style: a, b, c, … + Start at: 1 + Alignment: Left + Aligned at: 0.25" + Indent at: 0.5"

Formatted: Numbered + Level: 1 + Numbering Style: i, ii, iii, … + Start at: 1 + Alignment: Right + Aligned at: 0.75" + Indent at: 1" http://www.dodig.mil/hotline%20or%20800-424-9098 http://ctip.defense.gov/ mailto:ctipreports@osd.pentagon.mil http://www.polarisproject.org/

3.4.0CONTRACTOR PERFORMANCE REPORTING

1.4.1 Non-conformances

Audit or Inspection results that fall below contractual standards shall be identified as non-conformances.

1.4.1.1 Types of Non-conformance.

1.4.1.1.1 Minor Non-conformance: A minor non-conformance is a non-conformance, which by itself does not adversely impact mission, safety of personnel and/or equipment, performance (quality), schedule (delivery), or cost. A minor non-conformance normally does not increase risk to the Government.

2.4.1.1.2 Major Non-conformance: A major non-conformance is a non-conformance that adversely impacts mission, safety of personnel and/or equipment, performance (quality), schedule (delivery), or cost. A major non-conformance also increases risk to the Government. An example of increasing risk would be significant number of recurring non-conformances, which indicates inadequate preventive measures or actions, thus lowering the Government’s confidence that quality services will be provided on time and at cost.

2.4.1.2 Non-conformance Notification.

If a deficiency is found during inspection of the Contractor’s performance, the COR will determine the type of notification based on the category of the non-conformance (see Table 3). Non-conformances pre-identified by the contractor through the contractor’s QCP/internal audit does not require a govnerment non-conformance notice; however, if the Contractor fails to take corrective actions in a reasonable time-frame, the COR will notify the FSM, CO and QAPC for further actions.

1.4.1.2.1 First or Second Notices. As shown in Table 3, First and Second Notices provide written notification from the Government to the Contractor of minor non-conformances.

Initial minor non-conformances should be addressed with a First Notice to the applicable Contractor representative. The COR will notify the FSM, CO, and QAPC (ACCAMICQAPC@us.af.mil) of the issue, and prepare the notice for CO signature.

4.1.2.1.1. The COR will coordinate with the QAPC on the first or second notice NLT five business days after non-conformance notification, ensuring the first or second notice (attachment

1) is filled out correctly. The QAPC will provide an electronic copy of the coordinated first or second notice to the CO NLT 10 duty days after discovery of the event. If deemed appropriate, Table 3 - Non-conformance notifications

Non-conformance Category Notification Type Minor

Step 1: First Notice Step 2: Second Notice

Major Corrective Action Request (CAR)

Formatted: Outline numbered + Level: 1 + Numbering Style: 1, 2, 3, … + Start at: 3 + Alignment: Left + Aligned at:

0" + Indent at: 0.33", Tab stops: Not at 0.5"

Formatted: Outline numbered + Level: 2 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.25", Tab stops: Not at 1"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 4 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 2"

Formatted: Outline numbered + Level: 4 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 2"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 4 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

mailto:ACCAMICQAPC@us.af.mil the CO will issue the First and Second notice to the contractor with a suspense date for the contractors reply, sending a courtesy copy email to both the QAPC and COR.

4.1.2.1.2. The contractor shall send the completed first or second notice to the CO and QAPC (ACCAMICQAPC@us.af.mil). If the Contractor fails to respond by the specified due date, the QAPC will notify the CO for action and notify the COR to annotate the Contractor’s non-responsiveness in the CMS monthly services summary report. If the Contractor fails to resolve the issue as documented in the Notice, the CO may elevate the issue to the next level (Second Notice or CAR). Corrective/preventive actions initiated and/or completed by the Contractor will be verified by the COR on scheduled subsequent inspection and documented accordingly.

2.4.1.2.2 Corrective Action Requests (CAR) (see Attachment 2). As shown in Table 3, a

CAR provides written notification from the Government to the Contractor of significant performance discrepancies. The COR will recommend to the FSM, CO, and QAPC that a CAR be issued when previous notifications failed to result in corrective actions or when severe deficiencies exist. If a CAR is deemed necessary, the QAPC will assist the COR in filling out the CAR. The draft CAR will be routed NLT ten duty days after the event to the CO for review and signature, as the CO is the only person who has authority to issue a CAR to the Contractor.

4.1.2.2.1. To satisfactorily close-out a CAR, the following criteria must be met: the

Contractor must meet the suspense; the corrective action must have already begun; and the Contractor's actions must correct the deficiency and if applicable, be within specified standards.

4.1.2.2.2. If the corrective action is deemed UNSATISFACTORY by the CO, QAPC and COR, annotate the CAR accordingly and forward it to the CO with a brief explanation, including an estimate of the Government's loss due to the deficient performance. The estimate may account for loss or destruction of property, supplies, or equipment, and hours lost to re-perform the task. If the task cannot be re-performed, indicate the number of hours it ordinarily would have taken to do the task.

3.4.1.2.3 The COR will maintain a non-conformance log documenting all First and Second

Notices and CARs. The log will include the date sent to the Contractor, issue, due date for response and actual response date. If a detailed response is required, include a link to the response and include the response in the applicable COR folder tab.

2.4.2 COR Monthly Report

The COR will complete HQ ACC AMIC’s Contract Management System (CMS) Monthly COR Report.

1.4.2.1 Registration

The COR will request a login name and password at https://cms.acc.af.mil/ and complete the following:

First Name, Last Name and email address.

Formatted: Outline numbered + Level: 4 + Numbering

0" + Indent at: 0.5", Tab stops: Not at 2"

Formatted: Outline numbered + Level: 4 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 2"

Formatted: Outline numbered + Level: 2 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.25", Tab stops: Not at 1"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5" mailto:ACCAMICQAPC@us.af.mil https://cms.acc.af.mil/

User Role: Contracting Officer’s Representative

Justification: Primary COR on contract # FA4890-20-C-00XX

2.4.2.2 The Primary COR certifies the essential services accomplished by the contractor and documenting any issues or non-conformances that were identified during the month.

3.4.2.3 CMS will generate monthly emails to the primary COR(s) designated on the contract, monthly reports will be completed NLT the 10th calendar day of each month.

4.4.2.4 A primary COR will click on the embedded hyperlink in the email, or copy and paste the link onto their browser, OR Log into CMS, click on my requirements, then click on appropriate contract; click on Monthly COR Report.

5.4.2.5 The monthly report will display questions for the previous month (ex. answering questions in the month of March will display a monthly report for February)

1.4.2.5.1 Click on Answer Questions

2.4.2.5.2 Click on meets, above, below standards or N/A (see Table 4) and provide justification for each answer.

3.4.2.5.3 After completing the Monthly COR Report, click submit.

4.4.2.5.4 CMS will only allow one submitted report per month.

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Outline numbered + Level: 4 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 2"

Formatted: Outline numbered + Level: 4 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 2"

Formatted: Outline numbered + Level: 4 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 2"

Formatted: Outline numbered + Level: 4 + Numbering

Table 4- Monthly Performance Rating Definitions Rating Definition

Above Standards

Contractor performance exceeds contractual requirements and is beneficial to the government. To justify exceeds standards, state which PWS or SOW requirement(s) was exceeded (to include the paragraph number) along with how exceeding this requirement was a benefit to the government.

Meet Standards

Contractor performance meets contractual requirements. To justify meets standards, specify the PWS paragraph/requirement and state that the contractor met the requirement.

Below Standards

Contractor performance does NOT meet some contractual requirements. To justify below standard, identify an event(s) that the Contractor had trouble overcoming and state how it impacted the Government. This rating should be supported by referencing any non-conformance notification sent to the Contractor. Under Corrective Action, state what correction action the contractor has taken or is planning to take based on the contractor’s response to the non-conformation notification.

N/A Contractor was not contractually required to perform specific task or specific question is not applicable for evaluated period.

3.4.3 Annual Contract Performance Assessment Reporting System (CPARS)

An annual CPAR will be written on all Services contracts that exceed $1M total contract value. The Contractor Performance Assessment Reporting System (CPARS) https://www.cpars.gov/ is the Department of Defense (DoD) Enterprise Solution for collection of Contractor Past Performance Information (PPI) as required by the Federal Acquisition Regulation (FAR). CPARS is a web-enabled application that collects and manages a library of automated Contractor report cards. The COR will be designated as the Assessing Official Representative (AOR) in CPARS. The COR will utilize objective evidence from Monthly/Quarterly Reports, 1st/2nd notices, CARS, and any other information provided by SME’s that could be used for the CPAR assessment. The report will be due from the AOR within 20 calendar days following the end of the Period of Performance (PoP).

A CPARS quality checklist is available at the following link.

https://www.cpars.gov/pdfs/CPARSQualityChecklist.pdf. The table below is an excerpt from the AMIC CPARS Roles and Responsibilities Policy Letter.

Table 5- Performance Assessment Report Routine Timelines

CPAR Role: Timeframe Action in CPARS COR/AOR +20 calendar days following the end of the Period of Performance (PoP)

Draft (Narrative and rating) report and send to CO

CO +40 calendar days following the end of the PoP

Validate narrative/ratingcontract information and send to Contractor Representative

Formatted: Outline numbered + Level: 2 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.25", Tab stops: Not at 1" https://www.cpars.gov/ https://www.cpars.gov/pdfs/CPARSQualityChecklist.pdf

Contractor Representative 60 calendar days Comment and return to CO CO (Contractor Concur or does not sign)

7 calendar days but not to exceed PoP end +107 days

Completes CPAR

CO (Contractor Non-Concur) RO (Contractor Non-Concur

14 calendar days but not to exceed PoP end +114 days

CO forwards to RO; RO reviews, updates and completes CPARS

*Focal Point will notify AOR, CO/AO, or RO supervisor or division chief for all exceeded time frames

4.4.4 Procurment Integrated Enterprise Environment (PIEE) / Wide Area Work Flow

(WAWF).

The COR will accept or reject all Receiving Reports submitted by the Contactor in WAWF within 7 days of submittal by the Contractor. To access the receiving report, log into WAWF, select the Government, Acceptor, Acceptance Folder tab; then type in the contract and delivery order (contract/task order) number. The Shipment Number column in iRAPT is the Receiving Report number.

1.4.4.1 Validating Receiving Reports. The COR will review and validate all WAWF receiving reports (Shipment Number in iRAPT) ensuring that:

o• The Contractor submitted a combination invoice/receiving report o• The report reflects the correct contract and TO number o• Beginning and ending dates being billed are stated and accurate o• The correct CLIN(s) is(are) charged for billing period o• A description of the services is provided (Firm Fixed Price, Labor Hour, Travel, Materials o• The correct amount is charged, to include applicable handling rates o• Invoiced travel was pre-approved o• Back-up documentation is attached that includes a Contractor invoice with current and cumulative hours (for FFP LOE contracts) and amounts, as applicable o• A breakout of any other charges, as agreed to by the CO or required by the TO, is attached in iRAPT or has been provided to the COR

2.4.4.2 Certifying and Accepting Receiving Reports. Inspection of Services is ALWAYS performed by the appointed COR. The COR will validate and certify services through non-conformance notifications (para.4.1) and COR monthly reports (para.4.2.2.). After validation of services, the COR will select the “CQA” (Contract Quality Assurance) button to verify that the invoice was accurate, this step will normally select the Acceptance: button also. If anyone other than the COR is accepting services for the contract, the COR MUST certify in writing to the acceptor that services have been inspected/validated, the acceptor must then state in the comments block (under Misc Info tab) that the COR has reviewed, validated and certified services/invoices. In all cases, the COR will retain documentation (electronic or hard copy) showing COR validation and certification, as well as final acceptance of Contractor invoices.

Formatted: Outline numbered + Level: 2 + Numbering

0" + Indent at: 0.25", Tab stops: Not at 1"

Formatted: Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at:

0" + Indent at: 0.5", Tab stops: Not at 1.5"

Formatted: Indent: Left: 0.38", Outline numbered + Level:

2 + Numbering Style: Bullet + Aligned at: 0" + Indent at: 0"

Formatted: Indent: Left: 0.38", Hanging: 0.25", Outline numbered + Level: 2 + Numbering Style: Bullet + Aligned at:

0" + Indent at: 0"

Formatted: Outline numbered + Level: 3 + Numbering

3.4.4.3 Expenditure Log. After certification of an invoice in WAWF, the COR will update the expenditure log documenting/tracking certified invoice expenditures against funds allocated on the contract against each CLIN. Funding belongs to the organization where the Requirement Owner resides; the expenditure log allows the COR to ensure that funds are available for the work performed, to include travel and materials.

4.4.4.4 Rejecting Receiving Reports. If a receiving report is rejected in WAWF, the COR will go to the Miscellaneous Info tab to complete the comments block as follows:

a.• List details regarding all unsatisfactory services or incorrect invoice information

AND

b.• Instruct the Contractor to contact DFAS to have the invoice rejected back to the Contractor; this will allow the Contractor to make the same correction on both the receiving report and invoice, since combination invoices/receiving reports are required.

The rejecter will then return to the Header tab and click the “Reject to Initiator” block.

After rejecting a receiving report for unsatisfactory services, e-mail the CM/CO the contract number, receiving report number, and reason for rejection. Rejections for incorrect invoice information will be tracked by the COR.

4.5.0COR FOLDER.

The COR will establish and maintain a COR folder for each contract/task order; an electronic version of the folder may be kept, however, must be maintained in an area where both alternate CORs and COR Supervisor will have access to it. The COR folder must contain the tabs and contents outlined below in Table 7.