ECIS_Draft_Quality_Assurance_Surveillance_Plan_20210304.pdf

ATTACHMENT TBD ECIS QASP

12 February 21

Attachment (TBD)

N61340-21-R-0048

Quality Assurance Surveillance Plan (QASP)

Engineering Contractor Instructor Services (ECIS) Contract

1. Purpose

This Quality Assurance Surveillance Plan (QASP) is a Government developed and applied document used to make sure that systematic quality assurance methods are used in the administration of the standards included in this contract and in subsequent Option Years. The objective of the QASP is to ensure adequate quality controls and appropriate surveillance methods are in place to ensure the

Contractor, and that performance is in accordance with the requirements of the contract.

This QASP is intended to accomplish the following:

Define the roles and responsibilities of the Government officials responsible for implementation of the QASP.

Provide the Contracting Officer’s Representative (COR) with a guide to systematically and effectively assure performance of the Contractor.

Outline the corrective procedures to be taken against the Contractor for unacceptable or deficient performance.

2. Authority

Authority for issuance of this QASP is provided under Section E – Inspection and Acceptance of the contract. The Government quality assurance surveillance of the training services is to be executed by the

Contracting Officer or a duly authorized representative.

3. Scope

To fully understand the roles and responsibilities of the parties, it is important to first define the distinction in terminology between Quality Control Plan (QCP) and the QASP. The Contractor, and not the Government, is responsible for management and quality control actions necessary to meet the quality standards set forth by the contract and follow-on options. The QASP, on the other hand, is put in place to provide Government surveillance oversight of the Contractor’s efforts assuring those efforts are timely, effective and are delivering the results specified in the contract.

4. Government Resources

The following definitions for Government resources are applicable to this plan:

Contracting Officer - A person duly appointed with the authority to enter into (Procuring Contracting

Officer) (PCO) or administer (Administrative Contracting Officer) (ACO) contracts and make related determination and findings on behalf of the Government. The PCO for this contract is located at

NAWCTSD. The ACO will be designated in the contract. Contracting Officers are designated via a written warrant, which sets forth limitations of authority.

Contracting Officer Representative (COR) - An individual appointed in writing by the PCO to act as their authorized representative to assist in administering the contract. The COR will be appointed in the contract. The limitations of authority are contained in a written letter of appointment.

Attachment (TBD)

N61340-21-R-0048

5. Responsibilities

Contracting Officer – The Contracting Officer ensures performance of all necessary actions for effective contracting, ensures compliance with the terms and conditions of the contract, and safeguards the interests of the United States in the contractual relationship. It is the Contracting Officer that assures the

Contractor receives impartial, fair and equitable treatment under the contract. The Contracting Officer is ultimately responsible for the final determination of the adequacy of the Contractor’s performance.

The Contracting Officer is responsible for:

Contractor compliance with all requirements of the contract, contract administration, cost and property control.

Reviewing the COR assessment of the Contractor’s performance.

Administrative actions arising from unresolved differences between the COR and the Contractor.

Decisions that produce an increase or decrease in the scope of the contract or contract payment.

Evaluating the performance of the COR in monitoring PWS execution.

COR – The COR is responsible for technical administration of the contract and assures proper

Government surveillance of the Contractor’s performance. The COR is not empowered to make any contractual commitments or to authorize any changes on the Government’s behalf. Any changes that the

Contractor deems may affect contract price, terms or conditions shall be referred to the Contracting

Officer for action.

The COR is responsible for:

Verifying that the Contractor performs the technical requirements of the contract in accordance with the contract terms and conditions and PWS.

Reviewing, commenting, and reporting on the Contractor’s progress and ensuring the Contractor complies with reporting requirements.

Notifying the Contractor of discrepancies observed during surveillance and notifying the

Contracting Officer concerning performance of services provided by the contractor.

Maintaining contract records (including all Contract Data Requirements List (CDRL) submissions) delivered for their respective site(s).

Maintaining clear and effective lines of communication with the Contractor that comply (where applicable) with the requirements of the PWS.

Tracking invoice totals against contract ceiling to ensure contract ceiling is not exceeded.

Reviewing Contractor invoices and ensuring the proper labor categories and hours are charged and appear consistent with performance and are reasonable for the work performed.

Monitoring the contract for violations or deviations and keeping track of funds available and expended.

Ensuring the contract does not become a vehicle for personal services.

Coordinating site entry for Contractor personnel (if required) and ensuring that site specific check-in and check-out procedures are adhered to.

The QA monitoring forms (refer to Enclosures (1) and (2)) used to document the inspection and evaluation of the contractor’s work performance.

In addition to the above guidance, the following restrictions apply to the COR:

The COR cannot award or sign any contract or contract modification.

Attachment (TBD)

N61340-21-R-0048

The COR cannot obligate payment of monies.

The COR cannot change schedules (other than the daily training schedule), funds, or scope of the contract.

The COR cannot make any contractual agreements, commitments, or modifications involving price, quantity, quality, or delivery schedule.

The COR is not authorized or expected to offer advice to the Contractor or to Contractor personnel on how the work should be performed.

6. Methods of QA Surveillance

a. Observation/Inspection: The COR will monitor work accomplished and output in by using random sampling, observations, reviews, and inspections and will document the findings on a Quality Assurance

Monitoring Form (Enclosure 1). If a discrepancy is observed, the Quality Assurance Specialist will complete a Discrepancy Work Order (DWO) and forward it to the COR.

b. Customer Feedback: Customer satisfaction is the most significant external indicator of the success and effectiveness of all services provided and can be measured through customer complaints. The customer will communicate complaints to the COR as opposed to the contractor. The COR will accept the customer complaints and will investigate using the Quality Assurance Monitoring Form – Customer

Complaint Investigation identified in Enclosure (2).

c. Contractor Performance Assessment Report System (CPARS): The COR will conduct visits weekly during the reporting period to discuss any technical issues that may be prevalent with the

Contractor site manager. The COR will note these discussions and include a narrative assessment of overall Contractor performance and customer satisfaction annually in the CPARS. As this information may affect future source selections throughout DoD, the annual Government entry into CPARS will be used appropriately as a complementary performance oversight and communication tool with the QASP.

d. QASP: The method of surveillance detailed in the Contract Instructor Services Performance Metrics

(Enclosure 3) and the Data Performance Standards (Enclosure 4) shall be used by the COR in the technical administration of this QASP. In addition to the below instruction, the form to be used for documentation of the QA surveillance is the Surveillance Activity Checklist provided as Enclosure (5) herein.

7. Surveillance

Enclosures (1) and (2) provide a format for the COR to document inspection and evaluation of the

Contractor’s work performance. Enclosures (3) and (4) set forth the contract instructor services performance and data standards and surveillance methods for the Contractor and COR while Enclosure

(5) identifies the incentives to the Contractor. Enclosure (6) provides definitions of the evaluation areas and ratings that are used in CPARS. Enclosure (7) provides the checklist and means for the COR to document the results of the surveillance. The COR will evaluate, on a quarterly basis, the Contractor’s performance in meeting the identified contractor instructor services and data performance standards

(Enclosures (3) and (4)) utilizing their existing CPARS evaluation format and checklist. The culmination of these quarterly evaluations will result in the required annual CPARS reports. The Contractor’s reported performance on CPARS will be evaluated, as described in Enclosure (5), in the event the

Government determines to award an additional option year. Before awarding an option, the Government will review the Contractor’s CPARS to determine whether the Contractor’s performance met or exceeded the Government’s expectation for quality performance, unless another circumstance justifies an extended contractual relationship. Even if the Contracting Officer determines that awarding the option is

Attachment (TBD)

N61340-21-R-0048 advantageous to the Government after considering price, past performance, and other factors, the

Government is still free to decline award of another option year. Thus, the Contracting Officer is not required to consider these factors in declining to award an option. The factors are only relevant if the

Contracting Officer is awarding the option.

8. Documentation

In addition to providing quarterly reports to the Contracting Officer, the COR will maintain a complete

QA file. The file will contain copies of all reports, evaluations, recommendations, and any actions related to the Government’s performance of the QA function, including the originals of the Surveillance Activity

Checklists. All such records will be maintained for the life of the contract. The COR shall forward these records to the Contracting Officer at termination or completion of the contract.

At the end of each month, the COR will prepare a written report for the Activities Requirements

Originator (ARO) summarizing the overall results of the quality assurance monitoring of the contractor’s performance. This written report which consists of the contractor’s submitted monthly report and the completed Quality Assurance Monitoring Forms (Enclosure 1) will become part of the QA documentation. The ARO may require the contractor’s project manager, or a designated alternate, to meet with the ARO, COR, and other Government personnel as deemed necessary to discuss performance evaluation.

9. Enclosures:

(1) Quality Assurance Monitoring Form

(2) Quality Assurance Monitoring Form- Customer Complaint Investigation

(3) Contract Instructor Services Performance Metrics

(4) Data Performance Standards

(5) Incentives

(6) Definitions for Evaluation Areas/Ratings

(7) Surveillance Activity Checklist

(8) Annual Surveillance

Enclosure 1

Attachment (TBD)

QUALITY ASSURANCE MONITORING FORM

SERVICE or STANDARD:

SURVEY PERIOD: ___________________

SURVEILLANCE METHOD (Check): _____ Random Sampling

_____ 100% Inspection

_____ Periodic Inspection

_____ Customer Complaint

LEVEL OF SURVEILLANCE SELECTED (Check):

_____ Monthly

_____ Quarterly

_____ As needed

PERCENTAGE OF ITEMS SAMPLED DURING SURVEY PERIOD: ______ %

ANALYSIS OF RESULTS:

OBSERVED SERVICE PROVIDER PERFORMANCE MEASUREMENT RATE = ______%

SERVICE PROVIDER’S PERFORMANCE (Check): ____ Meets Standards

____ Does Not Meet Standards

NARRATIVE OF PERFORMANCE DURING SURVEY PERIOD:

PREPARED BY: __________________________________ DATE: _____________________

ENCLOSURE (1)

Enclosure 2

N61340-21-R-0048

QUALITY ASSURANCE MONITORING FORM –

CUSTOMER COMPLAINT INVESTIGATION

SERVICE or STANDARD:

SURVEY PERIOD: ___________________

DATE/TIME COMPLAINT RECEIVED: _____________ ________ AM / PM

SOURCE OF COMPLAINT: _______________________________ (NAME)

_______________________________ (ORGANIZATION)

_______________________________ (PHONE NUMBER)

_______________________________ (EMAIL ADDRESS)

NATURE OF COMPLAINT:

RESULTS OF COMPLAINT INVESTIGATION:

DATE/TIME SERVICE PROVIDER INFORMED OF COMPLAINT: _________________ AM / PM

CORRECTIVE ACTION TAKEN BY SERVICE PROVIDER:

RECEIVED AND VALIDATED BY:

PREPARED BY: __________________________________ DATE: _____________________

ENCLOSURE (2)

Enclosure 3

N61340-21-R-0048

CONTRACT INSTRUCTOR SERVICES

PERFORMANCE METRICS

PWS Task

Description

Performance Standard and Acceptable

Quality Level (AQL) Surveillance Method/Measure Incentives

Student Pass

Rate

The Contractor shall achieve a student pass rate of not less than 95% on each individual teacher and course.

100% Inspection by COR and customer feedback.

See enclosure

5.

Scheduled

Instructor

Hours

The Contractor shall achieve 100% of scheduled instructor hours per period of performance.

100% Inspection by COR

Management

Response

The Contractor shall provide timely responses to routine Government requests as described within the PWS and Appendices. The

Contractor shall provide timely responses to general Government requests for information or action within the timeframes requested.

100% Inspection by COR See enclosure

5.

Backup

Support

Execution

The Contractor shall achieve 100% of its backup support plan 100% Inspection by COR

See enclosure

5.

Great Lakes

Curriculum

Updates

The Contractor shall achieve 100% of the requested curriculum updates within the 3 business days from delivery to the Contractor.

100% Inspection by COR

Enclosure 3

ENCLOSURE (3)

Enclosure 4

DATA PERFORMANCE STANDARDS

Description Standard AQL Surveillance

Measure Incentives

Operational Security

(OPSEC) Plan

Submission is accurate and in accordance with the DID cited in the

CDRL.

95% acceptable on first submission to Government

100% acceptable on subsequent submission to

Government

100%

Inspection by

COR

See Enclosure 5

Conference Minutes

Submission is accurate and in accordance with the DID cited in the

CDRL.

95% acceptable on first submission to Government

100% acceptable on subsequent submission to

Government

100%

Mobilization CI

Contractor

Performance

Summary Report

Submission is accurate and in accordance with the DID cited in the

CDRL.

95% acceptable on first submission to Government

100% acceptable on subsequent submission to

Government

100%

Contractor

Performance

Summary Report

(CPSR)

Submission is accurate and in accordance with

100% acceptable on

Contractor Funding

Status Report

(CFSR)

Submission is accurate and in accordance with the DID cited in the

CDRL.

95% acceptable on first submission to Government

100% acceptable on subsequent submission to

Government

100%

Quality System Plan

Submission is accurate and in accordance with

100% acceptable on

Inventory Report

Submission is accurate and in accordance with

100% acceptable on subsequent submission to

Government

100%

Enclosure 4

N61340-21-R-0048

Testing and

Evaluation Report

Submission is accurate and in accordance with

100% acceptable on

Enclosure (4)

Enclosure 5

INCENTIVES

The following incentives shall apply to performance under this contract.

Assessment Period

Acceptable

Performance

Definition

How Measured Incentives

Base Period

06 Feb 2022 – 05 Jan 2023:

All measurement areas rated “Satisfactory.” See

Enclosure 4.

Endpoint of the assessment period ( 12th month) evaluation using CPARS format covering the previous 12 months.

(+) Meet the acceptable performance definition as a condition for award of future option periods.*

(-) Does not meet the acceptable performance definition as a condition for award of future option periods.*

Option Year 1

06 Jan 2023 – 05 Jan 2024:

period (24th month) evaluation using the CPARS format covering the previous 12 months.

Option Year 2

06 Jan 2024 – 05 Jan 2025:

period (36th month) evaluation

Option Year 3

06 Jan 2025 – 05 Jan 2026:

period (48nd month) evaluation

Option Year 4

06 Jan 2026 – 05 Jan 2027:

period (60th month) evaluation for award of future option periods.*

* The Government reserves the right to not award additional options even if all regulatory requirements are met and the Contractor meets the acceptable performance definition.

ENCLOSURE (5)

Enclosure 6

N61340-21-R-0048

DEFINITIONS FOR EVALUATION AREAS/RATINGS

All PWS/CDRL tasks, including PWS/CDRL sub-tasks, will be assessed focusing on the following.

Quality of Product or Service – Assess the Contractor’s effort to provide value-added products and/or services in meeting the performance work statement (PWS) requirements and transform operational needs and requirements into an integrated solution. This shall include activities to meet cost, performance, and schedule objectives. Assess how successfully the Contractor meets program value conditions.

Schedule – Assess the Contractor’s adherence to the required delivery schedule by assessing the

Contractor’s efforts during the assessment period that contribute to or effect the schedule variance. Also address significance of schedule disruptions, discuss causes, and assess the effectiveness of Contractor corrective actions.

Management –Discuss the extent to which the Contractor discharges its responsibility for coordination of all activity needed to execute the contract; identifies and applies resources required to meet schedule requirements; assigns responsibility for tasks/actions required by contract; communicates appropriate information to affected program elements in a timely manner. Assess the Contractor’s risk mitigation plans.

Other areas – Assess additional evaluation areas unique to the contract or that cannot be captured elsewhere.

The evaluation ratings are as follows:

Exceptional – Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the task and sub-task being assessed was accomplished with few minor problems for which corrective actions taken by the Contractor were highly effective.

Very Good - Performance meets contractual requirements and exceeds some to the Government’s benefit.

The contractual performance of the task and sub-task being assessed was accomplished with some minor problems for which corrective actions taken by the Contractor were effective.

Satisfactory - Performance meets contractual requirements. The contractual performance of the task and sub-task contain some minor problems for which corrective actions taken by the Contractor appear or were satisfactory.

Marginal - Performance does not meet contractual requirements. The contractual performance of the task and sub-task being assessed reflect a serious problem for which the Contractor has not yet identified corrective actions. The Contractor’s proposed actions appear only marginally effective or were not fully implemented.

Unsatisfactory – Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the task or sub-task contains a serious problem(s) for which the Contractor’s actions appear or were ineffective.

ENCLOSURE (6)

Enclosure 7

N61340-21-R-0048

SURVEILLANCE ACTIVITY CHECKLIST

The COR will perform a quarterly assessment of Quality, Schedule, Management, and Other Areas as applicable utilizing the CPARS evaluation rating definitions listed in Enclosure 6.

Quarterly Summary of Ratings:

Quality Schedule

Number of Exceptional ____ Number of Exceptional ____

Number of Very Good ____ Number of Very Good ____

Number of Satisfactory ____ Number of Satisfactory ____

Number of Marginal ____ Number of Marginal ____

Number of Unsatisfactory ____ Number of Unsatisfactory ____

Management Other Areas

Number of Exceptional ____ Number of Exceptional ____

Number of Very Good ____ Number of Very Good ____

Number of Satisfactory ____ Number of Satisfactory ____

Number of Marginal ____ Number of Marginal ____

Number of Unsatisfactory ____ Number of Unsatisfactory ____

Enclosure (7)

Description Surveillance

Method/Measure

Date

Planned

Date

Completed Quality Schedule Mgmt. Other

Success of Services

100% Inspection by COR of student pass rate and customer feedback.

TBD

Scheduled Instructor Hours 100% Inspection by COR

Operational Security (OPSEC) Plan

Contractor Performance Summary Report

(CPSR)

100% Inspection

Training Report

Quality Control (QC) Plan

Monthly Convening Report

Curriculum Recommendations Report

Testing and Evaluation Report

Instructor Schedule Report

Enclosure 8

N61340-21-R-0048

ANNUAL SURVEILLANCE

The COR will perform an annual assessment of overall contract performance utilizing the CPARS evaluation rating definitions listed in Enclosure 4 and the monthly summary of ratings. This annual assessment will occur at the endpoint of each period and cover the previous 12 months.

Description Surveillance

Method/

Measure

Date Planned Date

Completed

Summary

Rating/Results

Quality of Product or

Service

Review of Quarterly

Ratings.

Schedule

Management

Other Areas

Enclosure (8)