ATTACHMENT_04_MISTE_BASE_QUALITY_ASSURANCE_PLAN_R1_FINAL.pdf

Quality Assurance Plan (QAP) for

Modernizing Instrumentation Solutions for Test and Evaluation

(MISTE)

Indefinite Delivery / Indefinite Quantity Contract

# W900KK-19-R-0006

DEPARTMENT OF THE ARMY

PROGRAM EXECUTIVE OFFICE FOR SIMULATION, TRAINING, AND

INSTRUMENTATION

(PEO STRI)

Approved By: _________________________________ Date: _______________ Ron Miller, Program Manager ITTS IMO

MISTE Quality Assurance Plan (QAP) V.1 December 14, 2018 ii

Table of Contents

1. Introduction

2. Purpose

3. Scope

4. Performance Measurement and Monitoring Approach

5. Methods of Performance Monitoring

6. Communications and Customer Feedback

7. Customer Satisfaction

8. Formal Project Reviews

7. QASP Criteria Performance Metrics

8. Rating Definitions

9. Documenting Performance

10. Reviews and Resolution Appendix A – Acronym List Appendix B – Performance Report Appendix C – Non-Conformance Report (NCR)

1. Introduction The Modernizing Instrumentation Solutions for Test & Evaluation (MISTE) contract provides an Indefinite Delivery/Indefinite Quantity (ID/IQ) vehicle to the PEO STRI Instrumentation Management Office (IMO) for delivering test capabilities to the Army Major Range Test Facility Base (MRTFB) and DOD test & evaluation customers.

Individual Delivery Order (DO) work efforts may take the form of a Statement of Work (SOW), or Performance Work Statement (PWS), or Statement of Objectives (SOO). DO’s may be negotiated as Firm Fixed Price (FFP) or Cost Plus Fixed Fee (CPFF) or hybrid.

This overarching Quality Assurance Plan (QAP) was developed in accordance with Federal Acquisition Regulations (FAR) 46.4 “Government Contract Quality Assurance”, which states in 46.401(a) “Government quality assurance shall be performed at such times (including any stage of manufacture or performance of services) and places (including subcontractors’ plants) as may be necessary to determine that the supplies or services conform to contract requirements…Quality assurance surveillance plans should be prepared in conjunction with the preparation of the statement of work.”

2. Purpose This QAP describes a systematic method that will be used to evaluate the contractor’s performance under individual delivery orders. This QAP provides examples of how quality assurance will be monitored and measured by the DO Contracting Officer’s Representative (COR) and/or other assigned support staff. Examples include:

• What will be monitored, and the performance standards associated with that activity, including remediation/disposition actions

• How monitoring will take place

• Who will conduct the monitoring

• How monitoring efforts and results will be documented

• How will performance be rated

• How often will surveillance/measurements take place

The QAP sets forth the procedures and guidelines the PEO STRI staff will use in ensuring that contract performance standards are achieved through effective and ongoing monitoring of metrics and Acceptable Quality Levels (AQL). The performance standards associated with the majority of DOs can be found in Table 1 of Section 9 as the baseline for performance monitoring. Individual DOs will include the actual Quality Assurance Surveillance Plan (QASP) to be used under that DO. Adjustments or supplementation (additions, deletions, changes) to the monitoring criteria, metrics, or performance standards reflected in the QAP may be made based on the unique performance requirements of each individual DO.

The QAP performance standards (metrics) established herein are applicable to achieving the functional and operational standards, thresholds, and contract performance objectives of the DO SOW (or PWS / SOO as applicable) as well as the contractor’s effective project management of the effort. This QAP is designed specifically to effectively monitor the contractor’s ongoing performance to deliver a successful end-product that meets cost, schedule, and quality requirements. This QAP shall not be construed as a substitute for, or relieve the contractor of, the Government’s rights under FAR clauses 52.246-2 Inspection of Supplies—Fixed-Price or 52.246-3 Inspection of Supplies – Cost Reimbursement.

3. Scope The contractor is responsible for the quality control of all work performed in accordance with the performance standards set forth in the DO QASP. The contractor measures that quality through their own approved Quality Control Plan (QCP), while the Government’s role is ensuring proper Quality Assurance by measuring the performance standards of the QASP. The contractor’s QCP will be provided and accepted by the Government at the time of award and will be incorporated as a binding contractual requirement. As such, this QAP does not detail how the contractor will accomplish quality control in accordance with the performance standards provided herein. Rather it provides the performance standards and the Government’s approach to measurement and monitoring of those performance standards.

4. Performance Measurement and Monitoring Approach The Government’s performance measurement and monitoring (i.e. surveillance) approach defined herein provides the Government’s intentions and plans to review and analyze information generated thorough contractor performance assessments throughout the term of each specific DO according to DO QASP. The following Government personnel shall oversee and coordinate surveillance activities:

a. Contracting Officer (KO) - The KO shall ensure the performance of all necessary Government actions for effective contracting, compliance with the contract terms, and safeguard of the interests of the United States in the contractual relationship.

The KO shall also ensure that the contractor receives impartial, fair, and equitable treatment under this contract. The KO is ultimately responsible for the final determination of the adequacy of the contractor’s performance.

Assigned KO: Dan Veenstra

Organization or Agency: U.S. Army Contracting Command - Orlando

Telephone: 407-380-3968

Email: dan.veenstra.civ@mail.mil

b. Contract Specialist (CS) - The CS acts as an acquisition consultant and serves as liaison between the requesting program office, the contractor, and the supporting contracting office.

Assigned CS: Joseph Candelaria

Organization or Agency: U.S. Army Contracting Command - Orlando

Telephone: 407-208-3220

Email: joseph.candelaria5.civ@mail.mil

c. Contracting Officer’s Representative (COR) - The COR is responsible for technical administration of the contract and shall assure proper Government surveillance of the contractor’s performance. The COR shall keep a quality assurance file. At the conclusion of the contract, or when requested by the KO, the COR shall provide documentation to the KO. The COR is not empowered to make any contractual commitments or to authorize any contractual changes on the Government’s behalf. The contractor shall refer any changes they deem may affect contract price, terms, or conditions to the KO for action.

Assigned COR: TBD

Organization or Agency: PM ITTS-IMO, Telephone: TBD

Email: TBD

d. Government Product Manager (PM) – The Government PM is the customer of individual DOs under this contract. As such, the PM plays an important role in the customer feedback mechanism necessary for measurement of customer satisfaction for each specific DO. The focus of each DO is to fulfill requirements from a specific PM or PMs. Therefore, feedback from PMs through the mechanisms described in Sections 6-8 will provide the validation that requirements have been properly fulfilled.

Assigned PM: TBD

Organization or Agency: PM ITTS-IMO, Telephone: TBD

Email: TBD

e. The contractor has assigned the following contractor personnel to measure the contractor’s quality control against the performance standards of this QAP in accordance with the contractor’s QCP. In addition, they will serve as the contractor’s Program Manager and Task Manager for this contract.

Program Manager: TBD

Telephone: TBD

Email: TBD

Task Manager: TBD

Telephone: TBD

Email: TBD mailto:juan.b.orozco.civ@mail.mil

Other Contractor Personnel: TBD

Title: TBD

Telephone: TBD

Email: TBD

The Government’s acceptance of the contractor’s performance and deliverables will be based on analysis of factual data provided by the contactor and validated by the Government in accordance with the DO QASP. Based on this analysis, project performance information is documented, communicated, and used for contractor corrective actions, as applicable, and can be used as support for other contractual remedies that may be imposed by the Contracting Officer (KO) if performance is not maintained at acceptable levels in accordance with the performance standards of the DO

QASP.

Because the DO QASP defines the criteria and performance standards (i.e. metrics and AQLs) the contractor must continuously achieve for acceptable performance, it serves as the official basis for formal evaluation and rating of the contractor’s performance.

Accordingly, the QASP serves as the formal basis upon which the Contractor’s Performance Assessment Reporting System (CPARS) evaluation ratings are assigned in accordance with FAR 42.15.

5. Methods of Performance Monitoring The primary methods of DO performance surveillance include, but are not limited to:

a. Monitoring of contractor’s cost and schedule performance.

b. 100% review and inspection of all data and product deliverables including products physically installed at Government installations.

c. Periodic inspections of contractor facilities.

d. Direct observations of contractor work activities and deliverables.

e. Data validation.

f. Additional surveillance methods may be employed as determined necessary based on contractor performance. This could include assignment of an on-site Government quality assurance inspector/monitor and/or third party audits.

6. Communications and Customer Feedback The primary objective of ensuring a high level of effective communications is to ensure superior customer awareness at all times allowing for efficient customer feedback and leading to improved customer satisfaction. The timeliness and effectiveness of the contractor’s communications will be tracked, evaluated, and reported. Contractor’s communications will take the following forms:

a. Face-to-face meetings – as required

b. Reports – as required

c. Email – daily, as required

d. Briefings – every 30 days or earlier, if required

e. Teleconference – weekly or otherwise as scheduled

7. Customer Satisfaction Customer satisfaction is the most significant indicator of success and effectiveness of all products provided by the contractor. It will be evidence of proper contractor project management and quality control to ensure quality products are delivered on time and within cost. Customer feedback (level of satisfaction) will be primarily communicated in the form of email, verbally, and surveys using the communication methods in Section 6.

Customer satisfaction evaluation results will use the same definitions used for CPARS found in FAR 42.15, Table 42-1 “Evaluation Rating Definitions” and will be reflected under the “Management or Business Relations” factor in the annual assessment.

8. Formal Project Reviews Project reviews assist the Government customer in making ongoing business decisions regarding the project by evaluating its overall health at any given moment in time by:

a. Evaluating the effectiveness of project management

b. Comparing actual deliverables against the original requirements

c. Identifying lessons learned

d. Assessing performance (e.g. comparing the planned schedule, cost, and compliance requirements against the actuals)

e. Capturing stakeholders’ opinion of how the project was delivered

f. Disseminating project status results and findings for the record

7. DO QASP Criteria Performance Metrics The contractor is responsible for achieving the AQLs for all of the major processes identified below as included and identified under individual DO’s. The Government will perform surveillance of the contractor’s compliance with the standards through the various surveillance methods identified in Sections 5-8 of this QAP. The Government will document quality assurance findings on the formats provided in Appendix B – Non- Conformance Report (NCR) Performance Report herein. Table 1 below provides the standards, acceptable quality levels, method of surveillance, and remediation/disposition (incentives and disincentives) that will be imposed based on the contractor’s performance based on specific DO requirements adhering to the following criteria:

Table 1: Acceptable Quality Levels (AQL) NOTE: DO metrics and criteria may include additions, deletions, and revision to the baseline metrics shown below

Product/Service Requirement –

SOW Reference Performance

Standard Acceptable

Quality Level

(AQL)

Method of Surveillance Remediation/Disposition

1. Contractor Quality Control Plan (QCP) SOW 3.5.15

Full (100%) implementation and management of approved QCP accepted at time of contract award.

Compliance with approved QCP >98% of the time.

Periodic Inspection (minimum 4 times annually).

[I] When consistently met or exceeded annually, annual CPARS subfactor rating of Very Good or Exceptional.

[D] Standard not met 2 or more times annually, contractor considered contractually non-compliant. CAP required and annual CPARS subfactor rating no higher than Marginal. Other contractual remedies considered by Contracting Officer.

2. Data and Product Deliverables

SOW 3.8

and FAR clauses 52.246-2 Inspection of Supplies—Fixed- Price or 52.246-3 Inspection of Supplies – Cost Reimbursement

100% of deliverables are submitted timely and accepted by the Government upon initial submission.

>95% are accepted upon initial submission and are considered timely.

100% acceptable upon first resubmission.

100% review and inspection of all data and products.

[I] Contract compliance considered favorably for customer satisfaction and consideration of Option Exercise.

[D] Standard not met 3 or more times annually, Fixed Fee under CPFF reduced by 2% and Profit reduced by 3% under FFP. Potentially negative CPARS rating under Technical/Quality of Product or Service factor.

Product/Service Requirement –

SOW Reference Performance

Standard Acceptable

Quality Level

(AQL)

Method of Surveillance Remediation/Disposition

3.Contractor maintains adherence to

IMP/IMS SOW

3.4.2

Manage and track schedule status and accomplish all

IMP/IMS

milestones on schedule

100% schedule adherence demonstrated on all DO’s

IMP/IMS

tracking through monthly reports and status reviews.

[I] Contractor compliant with DO requirement and demonstrated good schedule control viewed favorably toward Government consideration of Option Exercise.

[D] Loss of schedule control and/or high variation is risk indicator requiring a CAP if schedule risk is imminent.

Consistently poor performance results in annotated CPARS narrative for Schedule/Timeliness and may result in rating of no higher than marginal if schedule product delivery date not met.

4. Integrated Performance Management System Cost and Schedule Performance tracked against EVM reporting

SOW 3.4.3

Strict adherence to negotiated contract cost and performance schedules.

> 95% cost control demonstrated on CPFF DO’s.

100% schedule adherence demonstrated on all DO’s.

Monthly Trend Analysis of EVMS data and other cost and schedule reporting requirements when EVM is applied.

[I] Consistently positive SPI (> 1) and CPI (> 1) indicators viewed favorably toward Government consideration of Option Exercise. When consistently met or exceeded, annual CPARS Cost Control and/or Schedule/ Timeliness factor rating of Very Good or Exceptional for CPFF DO’s.

[D] Consistent or significant negative SPI (<

1) and CPI (<1) indicators viewed unfavorably toward Government consideration of Option exercise. CAP required and annual CPARS Cost Control and/or Schedule/ Timeliness factor rating no higher than Marginal.

SOW Reference Performance

Standard Acceptable

Quality Level

(AQL)

Method of Surveillance Remediation/Disposition

5. Customer Satisfaction

No customer complaints with high levels of satisfaction reported.

Contractor has established positive working relationship with Government customer that is maintained.

A Corrective Action Plan (CAP) has been approved to the Government’s satisfaction within 15 calendar days after notification.

Results of ongoing dialog and satisfaction survey results.

[I] Consistently high levels of satisfaction viewed favorably toward Government consideration of Option exercise. When consistently high levels are maintained annually, CPARS Management or Business Relations factor rating of Very Good or Exceptional.

[D] Negative satisfaction levels viewed unfavorably toward Government consideration of Option exercise. CAP required.

Annual CPARS Management or Business Relations factor rating could be rated no higher than Marginal if CAP unsuccessful.

6. Program Management Reports and Reviews SOW

3.4 and 3.4.1

100% accuracy presented in contract performance and risk management status.

>98% accuracy in reporting that demonstrates program outcomes are being properly managed and controlled.

Effectiveness of Program Management Function is validated through survey methods, performance monitoring and communication and feedback.

[I] Standard consistently met is viewed favorably toward Government consideration of Option Exercise. When consistently high levels are maintained annually, annual CPARS Management or Business Relations factor rating of Very Good or Exceptional.

[D] Negative satisfaction levels viewed unfavorably toward Government consideration of Option exercise. CAP required.

Annual CPARS Management or Business Relations factor rating could be rated no higher than Marginal if CAP unsuccessful.

SOW Reference Performance

Standard Acceptable

Quality Level

(AQL)

Method of Surveillance Remediation/Disposition

7. Subcontractor Management

SOW 3.4.8

100% compliant with Small Business Participation percentages evaluated for contract award.

Subcontractor performance effectively monitored and controlled

>95% compliance on each delivery order

Monthly reporting and monitoring

[I] Contractor considered compliant for SBA reporting purposes.

Favorable CPARS ratings under Small Business Subcontracting factor under annual ratings when standard consistently met.

[D] Contractor considered non-compliant and consideration of Liquidated Damages equal to the percentage non-compliance in the form of Fee/Profit reduction may be imposed in addition to negative CPARS if standard not consistently met.

8. Rating Definitions

Table 2: FAR 42-1 Evaluation Ratings Definitions

Overall Performance Rating Standard

Exceptional

Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

Very Good

Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

Satisfactory Performance meets contractual requirements. The contractual performance of the element or sub-element contains some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

Marginal

Performance does not meet some contractual requirements. The contractual performance of the element or sub-element being evaluated reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented.

Unsatisfactory Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub-element contains a serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

9. Documenting Performance

a. Acceptable Performance The Government shall document positive (i.e. acceptable) performance. Any report may become a part of the supporting documentation for payments or other actions.

Additionally, surveillance results will be used to prepare or support Performance Assessment Reports in CPARS.

b. Unacceptable Performance When unacceptable performance occurs, the COR shall inform the contractor. This will normally be in writing unless circumstances necessitate verbal communication. In any case the COR shall document verbal communications via the COR monthly report in the COR tracking tool.

When the COR determines formal written communication is required, the COR shall prepare a Non-Conformance Report (NCR) and present it to the contractor’s task manager or on-site representative. A NCR template is attached to this QASP (See Appendix C – Non-Conformance Report (NCR) .

The contractor shall acknowledge receipt of the NCR in writing. The NCR will specify if the contractor is required to prepare a corrective action plan to document how the contractor shall correct the unacceptable performance and avoid a recurrence. The NCR will also state how long after receipt the contractor has to present his corrective action plan to the COR. The Government shall review the contractor’s corrective action plan to determine acceptability.

Any NCRs may become a part of the supporting documentation for contract payment deductions, fixed fee deductions, award fee nonpayment, or other actions deemed necessary by the KO.

Non-Conforming Performance. A Non-conformance occurs when the Contractor fails to meet the terms, conditions and/or specifications of the Contract/Task Order. All instances of non-conformance must be documented by the COR. The COR will provide the details of the non-conformance using the Nonconformance Report (NCR), to the KO, who will notify the Contractor of each nonconformance occurrence, unless the KO has authorized the COR to submit directly to the contractor, and copy the KO.

Resolution of any non-conformance discovered at the Subcontractor level will be addressed with the prime Contractor.

Non-conformances are classified as either: Level I (Minor); Level II (Major); or Level III (Critical).

Level I (Minor). Level I non-conformance is not likely to materially reduce the usability of services for their intended purpose or is a departure from established standards having little bearing on completing the contract requirement. Minor non-conformances can usually be corrected on the spot and can be issued to the Contractor through either a verbal or written Nonconformance Report (NCR). However, the Government is required to document verbal Level I NCRs. The COR may issue verbal Level I NCRs directly to the appropriate level of Contractor management. The COR will notify the KO as soon as practical when a verbal Level I NCR is issued and document them in their Monthly Reports. If the non-conformance cannot be corrected on the spot, the Contractor shall be given a suspense date to correct the deficiency. The KO will issue and sign written Level I NCRs.

Level II (Major). Level II non-conformance is likely to result in failure of the services to meet contract requirements. Also, repeated discrepancies, a large number of documented Level I NCRs and trends indicating failure of the Contractor’s quality control program can be classified as Level II NCRs. The KO issues all Level II NCRs in writing. The COR will notify the KO as soon as practical when a major non-conformance is identified.

Level III (Critical). Level III non-conformances result in hazardous or unsafe conditions due to the manner in which the services are performed; prevent or impact a vital agency mission as a result of performance; or are for repeated or uncorrected Level II NCRs.

The Contracting Office Director or Deputy Director issues written Level III NCRs. The COR is responsible for notifying the KO immediately upon discovery of a critical non-conformance.

Documenting NCRs. The COR will provide the details of the non-conformance using the NCR. If the Requiring activity prefers to issue a locally developed NCR, documentation must include as a minimum: the Contract or Task Order number;

reference to the specific contract requirement; the specific discrepancy to the requirement; where it was discovered; the date and time it was discovered; Contractor representative who was notified; and the suspense date for Contractor response/corrective action.

Tracking and Resolving NCRs. All NCRs, verbal and written, regardless of issuer will be included in the COR’s Monthly Report. The COR and KO shall track NCRs to ensure the Contractor identifies and takes appropriate corrective action to the root cause and the specific non-conformance is corrected within the required timeline. The KO will make a final written determination of the Government’s remedies if the non-conformance is not corrected by the Contractor after receiving notice and a reasonable opportunity to correct the work.

10. Reviews and Resolution The COR must initially coordinate and communicate with the contractor to resolve immediate performance issues or concerns while informing the Government’s PM and KO. The Government’s PM, COR, and contractor should jointly formulate and document long-term courses of action with the concurrence of the KO. Decisions regarding changes to metrics, thresholds, or service levels should be clearly documented and must be approved by the KO and incorporated into the contract via a modification.

If the contractor disputes a surveillance determination or has concerns over the Government’s performance management of the contract in general, they must first attempt to resolve through the Government’s PM. If, in the contractor’s opinion, resolution is not possible or there is clearly an impasse that is adversely affecting the contractual relationship and/or performance, the contractor must report their concerns to the KO within three business days from the date they determine resolution through the Government’s PM is not possible. The KO will be responsible to determining or negotiating the resolution approach and timeframe.

Appendix A – Acronym List

AQL – Acceptable Quality Level

CAP – Corrective Action Plan

COR – Contracting Officer’s Representative

CPARS – Contractor Performance Assessment Reporting System

CPFF – Cost Plus Fixed Fee

DO – Delivery Order

EVMS – Earned Value Management System

FAR – Federal Acquisition Regulations

FFP – Firm Fixed Price

KO – Contracting Officer

NCR – Non-Conformance Report

PWS – Performance Work Statement

PM – Project Manager

QASP – Quality Assurance Surveillance Plan

QCP – Quality Control Plan

SOO – Statement of Objectives

SOW – Statement of Work

Appendix B – Performance Report

1) CONTRACT NUMBER: W900KK-XX-D-XXXX

2) Prepared by: (Name of COR) [Insert Name]

3) Date and time of observation:

4) Description of Observation:

Examples of items to include in a report are:

− Method of surveillance:

− How frequently you conducted surveillance:

− Surveillance Results:

− Number of Observations:

Prepared by: [Enter COR’s name]

Signature of Contracting Officer’s Representative Date

Appendix C – Non-Conformance Report

Nonconformance Report (NCR): The COR may initiate a NCR at any time contractor performance is determined to be impacting cost, schedule, or performance standards. When the Contractor's performance is unsatisfactory, a NCR will be issued. The Contractor shall reply in writing within five (5) work days from the date of receipt of the NCR, giving the reasons for the unsatisfactory performance, corrective action taken, and procedures to preclude recurrence.

SECTION I

GENERAL INFORMATION

Contract No:

Contractor:

Contractor Department:

Location of Discrepancy:

NCR Control No:

Date Submitted:

PWS/PRS Paragraph #:

COR Name (print & sign):

Phone:

Date :

RESPONSE DUE:

Contractor Representative Acknowledgment:

(print, sign, & date)

SECTION II

DESCRIPTION OF NON-CONFORMANCE

TYPE OF NCR: MAJOR ___ MINOR ___

DESCRIPTION:

Contract/Modification No: NCR Control No:

SECTION III

ROOT CAUSE CORRECTIVE ACTION

Contractor Corrective Action:

Projected Completion/Implementation Date:

Contractor Representative/Date (print & sign):

Contract/Modification No: NCR Control No:

SECTION IV

FOLLOW-UP VERIFICATION

NCR Closed: YES ___ NO ___

COR Signature/Date:

Follow-up Comments: