Attachment 6 - Quality Assurance Surveillance Plan.pdf

HQ ACC ACQUISITION MANAGEMENT AND INTEGRATION CENTER (AMIC)

20 Oct 2021

Surveillance and Tracking Radar Processor (STRAP) Support Quality Assurance Surveillance Plan (QASP)

NOTICE: This Publication is available in digital format via the AMIC QA SharePoint.

OPR/Reviewed by: Reviewed by:

Mrs. Antoinea Noble Ms. Judy Seay Quality Manager (QM) Government Program Manager (G-PM)

ACC AMIC/DRQM ACC AMIC/PMCC

Received by:

Captain Desmond Awadzi Contracting Officer (CO)

ACC AMIC/PKFB

Revision: 0 Pages: 11 Distribution: F https://cs3.eis.af.mil/sites/AC-AQ-01-76/default.aspx

Contents

1.0 INTRODUCTION

2.0 AUTHORITY

3.0 AUDIT PROGRAM EXECUTION PROCESS

3.1 COR Limitations

3.2 COR Audits

3.3 Surveillance Documents

3.4 Assessing Performance

3.5 Procurement Integrated Enterprise Environment (PIEE)

4.0 COMBATING TRAFFICKING IN PERSONS (CTIP)

5.0 CONTRACTOR PERFORMANCE ASSESSMENT REPORT (CPAR)

TABLE 1 – SERVICE SUMMARY ITEMS

1.0 INTRODUCTION

This Quality Assurance Surveillance Plan (QASP) has been developed to provide simplistic instructions to aid you, the Contracting Officer’s Representative (COR), in performing oversight and surveillance activities of contractor performance. For additional guidance in performing your duties see the Defense Contingency COR Handbook given to you at training.

2.0 AUTHORITY

The Federal Acquisition Regulation (FAR) Clause 52.246 gives you the authority to inspect and test all supplies and services called for by the contract, to the extent practicable, at all places and times, including the period of manufacture, and in any event before acceptance or during the term of the contract/task order.

3.0 AUDIT PROGRAM EXECUTION PROCESS

The goal of all CORs is to ensure surveillance and measurement of contractor services is performed in accordance with (IAW) contract requirements with successful contract performance and mission accomplishment. The COR contribution is comprised of professional, non-adversarial relationships, which include positive and open communication with our customers and the contractor. The foundation of this relationship is built upon objective, fair, and consistent COR evaluations of contractor performance against contract requirements, the ability to discuss inspection results, trends, and items of mutual interest with the contract. Audits are performed using process/risk-based auditing techniques. Most people are familiar with using a checklist to perform some sort of inspection/audit. For the STRAP Support program, we use process-based and risk-based auditing to ensure we capture the most significant problems/highest risk to the program.

3.1 COR Limitations

CORs are the "eyes and ears" of the Quality Manager (QM), Program Manager (PM), and Contracting Officer (CO) relative to actual contract quality performance. Remember: CORs WILL NOT DIRECT WORK, CHANGE THE CONTRACT, OR FORMALLY INTERPRET THE CONTRACT. If there is an interpretation issue, contact the CO and PM. Refer to your designation letter for more detailed information.

3.2 COR Audits

It is essential that CORs accomplish sufficient in-depth audits in all areas to measure the quality of contractor performance and ensure the contractor is meeting contract requirements. This is where the Risk Management Plan and your previous audit results to include trends, recent contract changes, contractor manpower turnover, etc., come into play. This information (along with discussions with the QM and Functionals) should be used to determine what will be audited and to what depth. Is there really a need to audit Technical Orders every single time if you have not found any problems in the last two years? Probably not. Consider a sample of Technical Orders semi-annually. This should be ample and allow you to focus your audits on critical elements. Proper planning will ensure you audit the right requirements. Remember, as part of our audit preparation we will review the contractor’s documentation for trends and ensure the contractor is providing proper corrective and preventive actions. Additionally, compliance with contractor developed plans and regulations will be part of the surveillance, as they are deliverables or part of the contractor’s Quality Management System (QMS).

3.2.1 Types of Audits

CORs will be performing three types of audits: Initial Performance Assessment (IPA), audits, and obtaining customer feedback.

3.2.1 Initial Performance Assessment (IPA)

The IPA is a joint review by a multi-functional team (MFT) that may include the CO, PM, QM, CORs and subject matter experts (SMEs) to determine if the contractor has successfully completed transition and started performance, is fully operational, and is within the estimated cost, schedule, and performance parameters of the contract. The IPA will normally take place within 30 calendar days after the contractor assumes full performance responsibility (for your purposes - after contract start). The template is located on the ACC AMIC/DRQ SharePoint. Report contract performance as follows:

• Green--No issues.

• Yellow--Issue(s) but contractor has an adequate mitigation or corrective action plan in place.

• Red--Issue(s) with inadequate or no contractor mitigation or corrective action plan. Any “red” assessment shall include the Government proposed actions for the failing contractor.

3.2.1.1 Once Hundred Percent Inspection

This method will be used for Contractor submissions, products, and deliverables (tangible items). CORs will use the One Hundred Percent Inspection method to evaluate all reports delivered under paragraph

3.3. The COR, with assistance from the PM and CO if necessary, accepts or rejects these products based on the criteria and performance standards as outlined in the Services Summary. The COR also documents all results of inspections actions, and subsequent contractor re-performance taskings, if applicable

3.2.1.1 Periodic Audit

This type of surveillance consists of evaluating products/services not surveilled via one hundred percent inspection or by customer survey, and is implemented at a pre-defined frequency. An example of periodic audit is performing monthly inspections of class training conducted by the Contractor or weekly inspection of a Contractor’s process in developing training manuals.

3.2.1.2 Customer Survey or Customer Complaint

Although usually not a primary method, this is a valuable supplement to other, more systematic methods of surveillance. For example, in a case where random sampling indicates unsatisfactory service, customer complaints can be used as substantiating evidence. In certain situations, where customers complain consistently when the quality of performance is poor (e.g., dining facilities, custodial services), customer surveys and customer complaints may be a primary surveillance method, and customer satisfaction is an appropriate performance standard. In all cases, complaints will be documented, preferably on a standard form. All customer complaints will be forwarded to the PM. Information obtained from customer complaint activities may be used by the CO for actions relating to the Contractor.

3.3 Surveillance Documents

Title PWS Task Deliver to Due

Management and Staffing Plan 9.1 CO, PM and COR

-Initial: as part of contractor’s proposal;

-Subsequent Changes/Updates: due prior to contractor implementation

Transition-In Plan 3.1.2,

9.2 CO, PM and COR

-Initial: as part of contractor’s proposal;

-Pre-final: at Kick-off Meeting;

-Final: 5 calendar days after Kick-off Meeting

Kick-Off Meeting Agenda

3.1.3.2,

9.3 CO, PM and COR -5 calendar days after contract award

Kick-Off Meeting Minutes

3.1.3.4,

9.4 CO, PM and COR -5 calendar days after Kick-Off Meeting

Monthly Program Status Report

3.1.4, 3.3.1.3, 3.3.6.6.2.1, 5.1, 7.1.1.2, 9.5

CO, PM, COR

and AFCENT/A6

-30 calendar days after performance start and then on the 10th of every month

Title PWS Task Deliver to Due Meetings (Agenda, Minutes, Action Items)

3.1.5, 3.3.1.5, 9.6

COR and AFCENT/A6 -Within 5 calendar days of request

Data Calls 3.1.6, 9.7

COR and AFCENT/A6 -Within 1 business day of request

Cyber Security Workforce Report

3.1.8.3, 9.8

PM, COR and

AFCENT/A6

-30 calendar days after performance start and then on the 10th of every month

Transition-Out Plan 3.1.9.1,

9.9 CO, PM and COR

-Initial: within 6 months of performance start, then annually;

-During the final period of performance: submit quarterly, starting the first day of the final period of performance

Contract Close Out Report

3.1.10,

9.10 CO, PM and COR -No later than 30 calendar days after the contract completion date

Quality Control Plan 3.2,

9.11 PM and DRQ -Draft: 10 calendar days after contract award;

-Final: last day of the transition period

Technical Program/Project Reports

3.3.1.6, 3.3.2.3.1, 3.3.2.8, 3.3.3.6.3, 3.3.4.3, 9.12

COR and AFCENT/A6 -Within 5 calendar days of request

Trend Analysis Report 3.3.1.8, 9.13

COR and AFCENT/A6 -Within 5 calendar days of request

Engineering Designs

3.3.2.1, 3.3.2.2, 3.3.2.5, 3.3.2.6, 3.3.2.8, 3.3.3.6.3, 3.3.3.6.4, 9.14

COR and AFCENT/A6 -Within 7 calendar days of request

STRAP Integration Plan

3.3.2.2, 3.3.2.4, 3.3.3.6.2, 9.15

COR and AFCENT/A6 -Prior to on-site STRAP equipment integration

Testing and Evaluation

3.3.2.3, 3.3.2.7, 3.3.3.5, 3.3.3.6, 3.3.3.6.3, 9.16

COR and

AFCENT/A6

-Within 3 weeks of receipt of all data necessary for integration; and -Notify AFCENT/A6 upon completion

Installation/As-built Drawings

3.3.3.6.3, 3.3.3.6.4, 9.17

COR and

AFCENT/A6

-Within 10 calendar days of request; and -Within 30 days of project completion

Training Documentation

3.3.3.7.3, 9.18

COR and

AFCENT/A6

-Within 14 business days after completing each training session

Inventory Tracking Report

3.3.6.6.2.5, 3.3.6.6.3, 3.3.6.8.1, 4.2.8.

4.2.8.2, 9.19

CO, PM, COR

and AFCENT/A6

-Initial: 30 days after award;

-Subsequently: within 10 business days after request and Semi-Annually IAW FAR 52.245

Warranty Tracking 3.3.6.7, 3.3.6.7.1.

3.3.6.8, 9.20

DoD PIEE, CO and COR

PM, PLG and

COR

-At contract award upload in DoD PIEE; notify CO and COR when complete

-At time of delivery to AMIC PM, COR and PLG

Title PWS Task Deliver to Due

Failure Status Report 3.3.6.8.2, 9.21

COR and

AFCENT/A6

-Within 15 calendar days after the completion of the each quarter; quarters are based on the fiscal year beginning October

OCONUS Deployment Package

6.4,

9.22 PM and COR

-After receiving approval to travel, submit IAW DoD FCG lead-times for Theater and/or Country Clearance

System Security Plan (SSP) and Plans of Action (POAs)

8.12.1, 9.23

PM, COR and

AFCENT/A6

-Initially within 30 calendar days of government request;

-Subsequently within 15 calendar days after any changes or revisions

Security Assessment Plan/Reports

8.12.2, 9.24

PM, COR and

AFCENT/A6

-Initial: within 30 calendar days of government request;

-Subsequently: within 15 calendar after any changes or revisions; and -Within 15 calendar days of completion of a security assessment

3.4 Assessing Performance

Assess performance as a whole instead of individual findings and observations. Seldom do we pay for, nor can we afford perfection, so we should expect minor problems. From an auditing standpoint, we expect the problems to be isolated and not recurring (we expect corrective/preventive action). If we are finding a multitude of problems in the Logistics arena for example, it may indicate a systemic problem.

How does it affect the overall performance of the contract? What is the risk? Maybe these individual failures when looked at as a whole have caused a moderate or high risk. If this is the case, a Corrective Action Request (CAR) may be warranted. Remain objective, compare performance to contract requirements, look at only the objective evidence (not “hear say” unless you can corroborate it), and make solid decisions based on these facts.

3.4.1 Inspections

The COR may perform inspection/surveillance at any time during contract performance IAW FAR 52.246-2 – 5, as applicable. When planning audits, the COR /SME shall determine the intensity, frequency, and specific surveillance areas based on past performance results, potential risk, contract requirements, and other directed (mandatory) requirements. The process below is provided as a guide for CORs/SMEs to utilize when preparing to conduct an assessment, conducting the assessment, and reporting the results of that assessment to the PM, QM, and CO/CM. The COR will retain all audit and inspection documentation until contract closeout, unless otherwise approved by the CO.

3.4.1.1 Assess Performance – Describe how the process for assessing performance.

3.4.1.2 Evaluate Surveillance Documents para 3.3 and Service Summary Items in Table 1.

3.4.1.3 If assessing performance at more than one location, include the process for the Primary COR receiving the objective evidence for the audit.

3.4.1.4 Evaluate Combating Trafficking In Persons (CTIP). The COR will monitor the contractor’s performance regarding trafficking in persons such as that non-compliance with FAR clause 52.222-50, Combating Trafficking in Persons, is brought to the immediate attention of the contracting officer.

3.4.1 Non-conformances

Audit results that are below standards shall be identified as a nonconformance. Findings that generate a nonconformance shall be further classified using the following definitions/guidance:

3.4.2.1 Minor Nonconformance

A nonconformance that does not necessarily adversely impact mission, safety of personnel and/or equipment, environment, performance (quality), schedule (delivery), and/or cost. It is communicated to the contractor using two types of notification:

• First Notice or Second Notice (issued when the same minor nonconformance is identified again or the contractor fails to correct a first notice in a timely manner).

• First and Second notices will be controlled by using the (2-digit) Year/ (2-digit) Month / (3-digit) sequential number / (3-digit) Location / (1-digit) Identifier of “C” to identify it’s a CAR. (e.g., 1709001KUW-C)

• First and Second notices will be communicated to the CO and PM.

3.4.2.2 Major Nonconformance

A nonconformance that adversely impacts (or has the potential to) mission, safety of personnel and/or equipment, environment, performance (quality), schedule (delivery), and/or cost. This type of nonconformance has a risk assessment rating of moderate or high. Before deciding if it is a major, use the Risk Management Analysis Assessment Chart (5x5) in paragraph 2.4 to assist in determining risk (also located on the Non-conformance form). Major Non-conformances will be communicated to the contractor using a single type of notification:

• Corrective Action Request (CAR) – Major Nonconformance.

• When applicable, the QM and the COR shall prepare a CAR. The CAR will be controlled by using the (2-digit) Year/ (2-digit) Month / (3-digit) sequential number / (3-digit) Location / (1-digit) Identifier of “C” to identify it’s a CAR. (e.g., 1709001KUW-C).

• A CAR will be formally issued by the CO to the Contractor via a CO Letter. We can notify the contractor that a potential CAR is pending; however, it is not a CAR until the CO issues it as such.

3.5 Procurement Integrated Enterprise Environment (PIEE)

All components must use the Joint Appointment Module (JAM), within the Procurement Integrated Enterprise Environment (PIEE), to electronically track COR nominations, appointments, terminations, and training certifications for contracts assigned a COR. Components shall use the Surveillance and Performance Monitoring (SPM) Module, located in the PIEE, for all other contract surveillance actions.

3.5.1. In order to access SPM, the COR(s) must first register in PIEE and self-nominate in the Joint

Appointment Module (JAM). Training on how to access PIEE, register, and self-nominate will be provided by the AMIC QM during QAPC training.

3.5.2. Once the COR(s) has completed registration and been nominated, the COR(s) will maintain an electronic COR Surveillance file in the SPM Module for each contract assigned. This file must include, at a minimum;

3.5.2.1. A copy of the contracting officer's letter of designation and other documentation describing the COR's duties and responsibilities; and

3.5.2.2. Surveillance Checklist. The COR is responsible for developing standard procedures for Surveillance Checklists and may request QM assistance; the Surveillance Checklist serves as the guideline for all inspections. As a minimum, the Surveillance Checklist will include the items in the Services Summary and Deliverables Tables. The COR may also add other inspection activities/surveillance items to the Surveillance Checklist, with applicable PWS references, and obtain permission from the QM. These items may fluctuate throughout the course of the contract based on customer needs, identified risks, or issues related to Contractor performance. The COR will document all inspections conducted using the Surveillance Checklist and maintain them in the COR Surveillance file.

3.5.2.3. Monthly Report. The COR will perform audits/inspections and provide a report on contract performance to the contracting officer. These reports provide an opportunity to certify the completion of Services Summary items, as well as to document any exceptional work or any issues/non-conformances for the month. SPM monthly reports will be completed NLT the 5th calendar day of each month.

4.0 COMBATING TRAFFICKING IN PERSONS (CTIP)

As CORs one of the more important tasks is monitoring and reporting human trafficking compliance IAW FAR 22.17 and FAR 52.222-50/56, DFARS 222.17 and DFARS 252.222-7007, and PGI 222.17. If a CTIP compliance issue is suspected, ensure you report this to the QM, G-PM, or CO immediately or to any member in the member’s reporting chain. If for some reason, no one is available the DoD IG has a hotline and may be reached at (800) 424-9098, www.dodig.mil/hotline, or the National Human Trafficking Hotline (888) 373-7888. If additional information is required, the Department of State has a website/office dedicated to monitoring CTIP, https://www.state.gov/j/tip/. During quarterly validation of CTIP compliance, CORs will ensure at a minimum that the contractor provides a certified CTIP Compliance Plan after receiving award and the plan contains minimum the following:

1) An awareness program to inform contractor employees about the Government’s CTIP policy.

2) A recruitment and wage plan that only permits the use of recruitment companies with trained employees, prohibits charging recruitment fees to the employee, and ensures that wages meet applicable host-country legal requirements or explains any variance.

3) A housing plan, if the contractor or subcontractor intends to provide or arrange housing that ensures that the housing meets host-country housing and safety standards.

4) Procedures to prevent agents and subcontractors at any tier and at any dollar value from engaging CTIP violations.

During quarterly validation of CTIP compliance, CORs will also ensure at a minimum that contractors are not:

1) Engaging in trafficking in persons during the period of performance of the contract;

2) Procuring commercial sex acts during the period of performance of the contract;

3) Using forced labor in the performance of the contract;

4) Destroying, concealing, confiscating, or otherwise denying access by an employee to the employee’s identity or immigration documents, such as passports or drivers’ licenses, regardless of issuing authority;

5) Using misleading or fraudulent practices during the recruitment of employees or offering of employment. Using recruiters that do not comply with local labor laws of the country in which the recruiting takes place;

6) Charging employees recruitment fees;

7) Failing to provide return transportation or pay for the cost of return transportation upon the end of employment or failing to provide return transportation or pay for the cost of return transportation upon the end of employment

8) Providing or arranging housing that fails to meet the host country housing and safety standards; or

9) Failing to provide an employment contract, recruitment agreement, or other required work document in writing, if required by country labor laws.

5.0 CONTRACTOR PERFORMANCE ASSESSMENT REPORT (CPAR)

http://www.dodig.mil/hotline http://www.state.gov/j/tip/

This is a critical document giving the CO, PM, and QM performance oversight on all services and ensures all products during the year were provided to the Government in accordance with all contract requirements. When notified to provide information on the contractor’s performance, the G-PM will suspense the QM. The PM will determine when the CPAR is due

TABLE 1 – SERVICE SUMMARY ITEMS

SS# Performance Objective PWS Paragraph Performance Threshold

Provide key personnel at contract start; AND Notify the government within 24 hours of key personnel vacancy;

AND

Replace key personnel no later than 30 calendar days from start of vacancy

7.1

Performance is acceptable when:

100% of key personnel are in place at contract start; AND the mission is not impacted due to key personnel vacancies

2 Develop and submit deliverables 9.0

Performance is acceptable when:

98% of deliverable requirements are met and received on time (not counting excusable delays); AND Deliverables are complete and have no flawed information, analyses, assumptions or conclusions

Provide logistics and supply services meeting standards of quality and workmanship.

3.3.5 3.3.6

Performance is acceptable when:

Performance meets PWS and/or design requirements;

Only minor problems/exceptions exist, or the contractor recovered major problems or exceptions without a negative impact to the contract or mission; AND No significant weaknesses/deficiencies were identified.

All aspects of equipment installations or removals conform to standards.

3.3.2 3.3.3 3.3.4

Performance is acceptable when:

Performance meets PWS and/or design requirements;

Only minor problems/exceptions exist, or the contractor recovered major problems or exceptions without a negative impact to the contract or mission; AND No significant weaknesses/deficiencies were identified.

All functionality of the product/program works as designed with a maximum of 5% of minor problems/minor exceptions. (Products relates to coding during development, modeling, simulation, etc.)

3.3.2 3.3.3 3.3.4

Performance is acceptable when:

Performance meets PWS and/or design requirements;

Only minor problems/exceptions exist, or the contractor recovered major problems or exceptions without a negative impact to the contract or mission; AND No significant weaknesses/deficiencies were identified.

Provide integration planning, evaluation, testing, and maintenance IAW established milestones.

3.3.2 3.3.3 3.3.4

Performance is acceptable when:

Delivery milestones are met (not counting excusable delays);

Only minor problems/exceptions exist, or the contractor recovered major problems or exceptions without a negative impact to the contract or mission; AND No significant weaknesses/deficiencies were identified.

SS# Performance Objective PWS Paragraph Performance Threshold

Accurately forecast, manage and control costs, and monitor cost variances.

3.0 5.1

Performance is acceptable when:

Actual cost do not exceed original budgeted/funded cost per performance period and the life of the contract; AND Projected and actual overruns are communicated to the government, in writing requirements.

Customer satisfaction; customer satisfaction is based on positive or negative responses from customers.

(Does not include policy-related comments)

Performance is acceptable when:

a) No negative customer complaints are received (written or verbal) from O6 or civilian equivalent.