Attachment_5_-_SVED_Quality_Assurance_Surveillance_Plan.pdf

Quality Assurance Surveillance Plan (QASP)

AEC SVED

Contract W91CRB-18-R-0037

Dated: 4 April 2018

1 Vision

1.1 Mission

3 Purpose

4 Authority

5 Roles and Responsibilities

5.1 Contracting Officer (KO)

5.2 Contracting Officer's Representative (COR)

5.3 Other Key Government Personnel

5.4 Contractor Representatives

5.5 Program/Project Manager

6 Performance Requirements and Method of Surveillance

6.1 Contract Surveillance

6.2 Surveillance Matrix

Appendix 1 - Surveillance Matrix

Appendix 2 - Corrective Action Report (CAR)

Appendix 3 - Customer Complaint Record

Appendix 4 - Performance Assessment Report (PAR)

Quality Assurance Surveillance Plan (QASP)

1 Vision

Support the Warfighter with effective, suitable, and survivable evaluations of systems.

1.1 Mission

AEC plans, coordinates and executes the Army’s independent evaluations to provide essential information to decision makers.

3 Purpose

This QASP is a Government developed document used to determine if the Contractor's performance meets the performance standards contained in the contract. The QASP establishes procedures on how this assessment/inspection process will be conducted.

It provides the detailed process for continuous oversight:

What will be monitored

How monitoring will take place

Who will conduct the monitoring

How monitoring efforts and results will be documented

The Contractor is responsible for implementing and delivering performance that meets contract standards using its Quality Control Plan. The QASP provides the structure for the Government's surveillance of the Contractor's performance to assure that it meets contract standards. It is the Government's responsibility to be objective, fair and consistent in evaluating Contractor performance.

The QASP is not part of the contract nor is it intended to duplicate the Contractor's quality control plan. This QASP is a living document. Flexibility in the QASP is required to allow for an increase or decrease in the level of surveillance necessary based on Contractor performance.

The Government may provide a copy of the QASP to the Contractor to facilitate open communication. In addition, the QASP should recognize that unforeseen or uncontrollable circumstances might occur that are outside the control of the Contractor.

The QASP should ensure early identification and resolution of performance issues to minimize impact on mission performance.

4 Authority

Authority for issuance of this QASP is provided under Part 46 of the Federal Acquisition Regulation, Inspection of Services clauses, which provides for inspection, acceptance and documentation of the service called for in the contract or order. This acceptance is to be executed by the Contracting Officer or a duly authorized representative.

5 Roles and Responsibilities

The following personnel shall oversee and coordinate surveillance activities:

5.1 Contracting Officer (KO)

The KO shall ensure performance of all necessary actions for effective contracting and compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The KO shall also ensure the Contractor receives impartial, fair, and equitable treatment under this contract. The KO shall determine and document the final assessment of the Contractor's performance in the Government’s past performance tracking system.

Assigned KO: Gerald H. Taulbee

Organization or Agency: U.S. Army Acquisition Command

Telephone: 410-306-2851

Email: gerald.h.taulbee.civ@mail.mil

5.2 Contracting Officer's Representative (COR)

The COR is responsible for providing continuous technical oversight of the Contractor's performance. The COR uses the QASP to conduct the oversight/surveillance process.

The COR shall keep a Quality Assurance file that accurately documents the Contractor's actual performance. The purpose is to ensure the Contractor meets the performance standards contained in the contract. The COR is responsible for reporting early identification of performance problems to the KO. The COR is required to provide an annual performance assessment to the KO which will be used in documenting past performance. The QASP is the primary tool for documenting Contractor performance.

The COR is not empowered to make any contractual commitments or to authorize any contractual change on the Government's behalf.

Assigned COR: Dawn Sauer

Organization or Agency: U.S. Army Test and Evaluation Command

Telephone: 410-306-1495

Email: dawn.r.sauer.civ@mail.mil

5.3 Other Key Government Personnel

(enter name or delete these lines if not applicable) This may include performance monitors, inspectors, technical experts, or others who provide information that helps the COR monitor Contractor performance.

5.4 Contractor Representatives

The following employees of the Contractor serve as the Contractor's Program Manager and Task Manager for this contract. (Complete this section after the contract award)

Program Manager: <upon award, enter name>

Telephone: <enter number>

Email: <enter address>

Task Manager - <upon award, enter name>

Telephone: <enter number>

Email: <enter address>

Other Key Contract Personnel: <upon award, enter name or delete these lines if not applicable>

Title: <enter title>

Telephone: <enter number>

Email: <enter address>

5.5 Program/Project Manager

The PM provides primary program oversight, nominates the COR, ensures the COR is trained before performing any COR duties, and supports the COR's performance assessment activities. While the PM may serve as a direct conduit to provide Government guidance and feedback to the Contractor on technical matters, they are not empowered to make any contractual commitments or any contract changes on the Government's behalf.

Assigned PM: <enter name>

Organization or Agency: <enter organization or Agency name>

Telephone: <enter number>

Email: <enter address>

6 Performance Requirements and Method of Surveillance

This section describes the special requirements for this effort. The following sub-sections provide details of various considerations on this effort.

6.1 Contract Surveillance

The goal of the QASP is to ensure that Contractor performance is effectively monitored and documented. The COR's contribution is their professional, non-adversarial relationships with the KO, PM and the Contractor, which enables positive, open and timely communications. The foundation of this relationship is built upon objective, fair, and consistent COR evaluations of Contractor performance against contract requirements. The COR uses the methods contained in this QASP to ensure the Contractor is in compliance with contract requirements. The COR function is responsible for a wide range of surveillance requirements that effectively measure and evaluate the Contractor's performance. Additionally, this QASP is based on the premise that the Contractor, not the Government, is responsible for management and QC/QA actions to successfully meet the terms of the contract.

6.2 Surveillance Matrix

The Surveillance Matrix is the list of performance objectives and standards that must be performed by the Contractor. This matrix details the method of surveillance the COR will use to validate and inspect these performance elements. Inspection of each element will be documented in the COR file.

Performance objectives define the desired outcomes. Performance Standards define the level of service required under the contract to successfully meet the performance objective. The inspection methodology defines how, when, and what will be assessed in measuring performance. The Government performs surveillance, using this QASP, to determine the quality of the Contractor's performance as it relates to the performance element standards. The Program Requirements Summary should be used to form the foundation of the COR's inspection checklist.

In evaluating the quality of the Contractor’s performance, the following performance ratings may be used.

Performance Rating

Criteria

Excellent / Outstanding

Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the element or sub-element being assessed was accomplished with no problems and Contractor actions were highly effective.

Very Good Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance was accomplished with few minor problems for which corrective actions taken by the Contractor were effective.

Good Performance meets contractual requirements. The contractual performance contains some minor problems for which corrective actions taken by the Contractor were satisfactory.

Marginal Performance does not meet some contractual requirements.

The contractual performance reflects a serious problem for which the Contractor has not yet identified corrective actions.

The Contractor’s proposed actions appear marginally effective or were not fully implemented.

Unsatisfactory Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance contains serious problem(s) for which the Contractor’s corrective actions appear or were ineffective.

Appendix 1 - Surveillance Matrix

Statements Standards/AQLs Inspections Ratings

3.1 The Contractor shall support planning, development, and execution efforts for test and evaluation.

Deliverables:

A001 Progress and Status Report A002 Project calendar A003 Reports and Briefings A004 Post Meeting Report A005 Draft Input to Test and Evaluation Documents A006 Draft Input to Test and Evaluation Planning Documents A007 Mission Scenarios A008 Test Observation and Data Collection A009 Observations and lessons learned A015 Documentation Repositories

a) Tasks assigned in support of test and evaluation planning shall be accurate to the system, test, evaluation, and Government Subject Matter Expert (SME) documentation and intent.

AQL: Acceptable performance of task requires the Contractor to meet at a minimum 95% accuracy and completeness with correction to 100% upon identification of deficiencies, executed within COR specified timelines.

b) Provide comprehensive documentation for assigned tasks.

AQL: Acceptable performance of task requires the Contractor to meet at a minimum 95% accuracy and completeness with correction to 100% upon identification of deficiencies, executed within COR specified timelines.

c) Identify any deficiencies associated with the test and/or evaluation planning.

AQL: Acceptable performance of task requires the Contractor to meet at a minimum 95% accuracy and

What: Support to Test and Evaluation (T&E) How: As needed, the COR with assistance from the Government SME will inspect Contractor deliverables and work status.

Who: The Contracting Officer Representative (COR) in collaboration with the Government Subject Matter Expert (SME).

Incentive: If the support for planning of the test and evaluation is not complete, is incorrect or is unsatisfactory, the Contractor will develop a cure plan to avoid further substandard work and re-perform the work at no additional cost or fee to the Government.

Standard(s): Inspection applies to all standards completeness with correction to 100% upon identification of deficiencies, executed within COR specified timelines.

d) Maintain communication with COR and Government SMEs and ensure products are coordinated with other program activities.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and completeness by the COR specified timelines.

e) Completes actions within specified deadlines.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and completeness by the COR specified timelines.

f) Conforms with regulations and pamphlets (DOD, Army, ATEC) and other references as defined in task orders.

AQL: Acceptable performance of task requires the Contractor to meet at a minimum 95% accuracy and completeness with correction to 100% upon identification of deficiencies, executed within COR specified timelines.

3.2 The Contractor shall provide database and data management support for tests and evaluations.

Deliverables:

A001 Progress and Status Report A003 Reports and Briefings A012 Data Documentation and Architecture A013 Data Management Tools A014 Data Analysis and Visualization

a) ATEC specified tools.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and completeness by the COR specified timelines.

b) Completes actions within specified deadlines.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and completeness by the COR specified timelines.

c) Processes shall be repeatable with consistent results.

AQL: Acceptable performance of task requires the Contractor to meet at a minimum 95% accuracy and completeness with correction to 100% upon identification of deficiencies, executed within COR specified timelines.

d) Must be ATEC supportable.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and completeness by the COR specified timelines.

e) If database is connected to the network, meets DA compliance.

AQL: Acceptable performance of task requires the Contractor to meet

What: Databases and data management work How: As needed, the COR with assistance from the Government SME will inspect Contractor deliverables and work status.

Who: The Contracting Officer Representative (COR) in collaboration with the Government Subject Matter Expert (SME).

Incentive: If the work or support is not completed as specified, according to standards, is incorrect or is unsatisfactory, the Contractor will develop a cure plan to avoid further substandard work and re-perform the work at no additional cost or fee to the Government.

at a minimum 95% accuracy and

100% upon identification of deficiencies, executed within COR specified timelines.

f) Data authentication must be in accordance with DAGE operations described in DA PAM 73-1 (para 6- 50).

AQL: Acceptable performance of task requires the Contractor to meet at a minimum 95% accuracy and completeness with correction to 100% upon identification of deficiencies, executed within COR specified timelines.

g) Analysis shall accurately capture findings from data.

AQL: Acceptable performance of task requires the Contractor to meet at a minimum 95% accuracy and completeness with correction to 100% upon identification of deficiencies, executed within COR specified timelines.

h) Conforms with regulations and pamphlets (DOD, Army, ATEC) and other references as defined in task orders.

AQL: Acceptable performance of task requires the Contractor to meet

100% upon identification of specified timelines.

i) Maintain communication with COR and Government SMEs and ensure products are coordinated with other program activities.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and completeness by the COR specified timelines.

3.3 The Contractor shall conduct evaluation, analytical, and verification support for tests, evaluations, and special projects.

Deliverables:

A001 Progress and Status Report A003 Reports and Briefings A009 Observations and lessons learned A011 Verification, Validation and Accreditation Documents A014 Data Analysis and Visualization

a) Completes actions within specified deadlines.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and completeness by the COR specified timelines.

b) ATEC specified format.

AQL: Acceptable performance of task requires the Contractor to meet at a minimum 95% accuracy and completeness with correction to 100% upon identification of deficiencies, executed within COR specified timelines.

c) Accurately represents analysis and meets professional standards.

AQL: Acceptable performance of

What: Evaluation, analysis, verification, reporting and associated work How: As needed, the COR with assistance from the Government SME will inspect Contractor deliverables and work status.

Who: The Contracting Officer Representative (COR) in collaboration with the Government Subject Matter Expert (SME).

Incentive: If the work or support is not completed as specified, according to standards, is incorrect or is unsatisfactory, the Contractor will develop a cure plan to avoid further substandard work and re-perform the work at no additional task requires the Contractor to meet completeness with correction to 100% upon identification of cost or fee to the Government.

3.4 The Contractor shall assist, conduct, and develop new and refined methodologies, studies, and analysis for tests and evaluations.

Deliverables:

A001 Progress and Status Report A003 Reports and Briefings A016 Policy Refinements A017 Guidelines and Standards A018 Methodologies and processes

a) Policies can be implemented and executed.

AQL: Acceptable performance of task requires the Contractor to meet at a minimum 95% accuracy and completeness with correction to 100% upon identification of deficiencies, executed within COR specified timelines.

b) Conforms with regulations and pamphlets (DOD, Army, ATEC) and other references as defined in task orders.

AQL: Acceptable performance of task requires the Contractor to meet at a minimum 95% accuracy and completeness with correction to 100% upon identification of deficiencies, executed within COR specified timelines.

c) Completes actions within specified deadlines.

AQL: Acceptable performance of task requires the Contractor to

What: Methodologies, studies, analysis and associate work How: As needed, the COR with assistance from the Government SME will inspect Contractor deliverables and work status.

Who: The Contracting Officer Representative (COR) in collaboration with the Government Subject Matter Expert (SME).

Incentive: If the work or support is not completed as specified, according to standards, is incorrect or is unsatisfactory, the Contractor will develop a cure plan to avoid further substandard work and re-perform the work at no additional cost or fee to the Government.

achieve 100% accuracy and timelines.

d) Identifies and implements effective methodologies.

AQL: Acceptable performance of task requires the Contractor to meet at a minimum 95% accuracy and completeness with correction to 100% upon identification of deficiencies, executed within COR specified timelines.

e) Implementable and executable.

AQL: Acceptable performance of task requires the Contractor to meet at a minimum 95% accuracy and completeness with correction to 100% upon identification of

3.5 The Contractor shall assist, develop, and conduct scientific and technical services for test and evaluation.

Deliverables:

A001 Progress and Status Report A003 Reports and Briefings A007 Mission Scenarios A009 Observations and lessons

a) Completes actions within specified deadlines.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and completeness by the COR specified timelines.

b) Documents meet ATEC style and format requirements as specified in ATEC Regulation 73-1.

What: Scientific and technical services and associate work How: As needed, the COR with assistance from the Government SME will inspect Contractor deliverables and work status.

Who: The Contracting Officer Representative (COR) in collaboration with the Government Subject Matter Expert (SME).

learned A010 Models, Simulations and Stimulations A011 Verification, Validation and Accreditation Documents A014 Data Analysis and Visualization A019 Training Materials

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and completeness by the COR specified timelines.

c) Modeling and simulation work is in accordance with ATEC PAM 73- 21, ATEC regulation 73-21, Department of Army PAM 5-11.

AQL: Acceptable performance of task requires the Contractor to meet at a minimum 95% accuracy and completeness with correction to 100% upon identification of

Incentive: If the work or support is not completed as specified, according to standards, is incorrect or is unsatisfactory, the Contractor will develop a cure plan to avoid further substandard work and re-perform the work at no additional cost or fee to the Government.

3.6 The Contractor shall support test and safety management for activities.

Deliverables:

A001 Progress and Status Report A003 Reports and Briefings A004 Post Meeting Report A005 Draft Input to Test and Evaluation Documents A006 Draft Input to Test and Evaluation Planning Documents A008 Test Observation and Data Collection

a) Completes actions within specified deadlines.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and completeness by the COR specified timelines.

b) AR 385-10, DA PAM 385-16, DA PAM 385-30, MIL-STD 882E and other appropriate regulations as applicable.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and

What: Test and safety management activities and associated work How: As needed, the COR with assistance from the Government SME will inspect Contractor deliverables and work status.

Who: The Contracting Officer Representative (COR) in collaboration with the Government Subject Matter Expert (SME).

Incentive: If the work or support is not completed as specified, according to standards, is incorrect

A009 Observations and lessons learned A015 Documentation Repositories A016 Policy Refinements A017 Guidelines and Standards A018 Methodologies and processes completeness by the COR specified timelines.

c) ATEC specified format.

AQL: Acceptable performance of task requires the Contractor to meet at a minimum 95% accuracy and completeness with correction to 100% upon identification of deficiencies, executed within COR specified timelines.

d) Maintain accurate progress and status reports.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and completeness by the COR specified timelines.

e) AEC specified tools.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and completeness by the COR specified timelines.

f) Maintain communication with COR and Government SMEs and ensure products are coordinated with other program activities.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and or is unsatisfactory, the Contractor will develop a cure plan to avoid further substandard work and re-perform the work at no additional cost or fee to the Government.

3.7 The Contractor shall prepare progress and labor reporting documents as specified.

Deliverables:

A001 Progress and Status Report

a) Completes actions within specified deadlines.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and completeness by the COR specified timelines.

b) Maintain accurate progress and status reports.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and completeness by the COR specified timelines.

c) AEC specified tools.

AQL: Acceptable performance of task requires the Contractor to achieve 100% accuracy and

What: Progress and labor report How: As needed, the COR with assistance from the Government SME will inspect Contractor deliverables and work status.

Who: The Contracting Officer Representative (COR) in collaboration with the Government Subject Matter Expert (SME).

Incentive: If the work or support is not completed as specified, according to standards, is incorrect or is unsatisfactory, the Contractor will develop a cure plan to avoid further substandard work and re-perform the work at no additional cost or fee to the Government.

Appendix 2 - Corrective Action Report (CAR)

CORRECTIVE ACTION REPORT (CAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACTOR

2. CONTRACT NUMBER

3. TYPE OF SERVICES

4. FUNCTIONAL AREA

5. SUSPENSE DATE

6. CONTROL NUMBER

7. DEFICIENCY MAJOR MINOR

FINDING:

FINDING IMPACT:

Please respond with a written corrective action plan that details the corrective action of the cited deficiency, the cause of the deficiency, and actions taken to prevent recurrence by Suspense Date in Block 5. If date was not entered in Block 5, the contractor is not required to provide a response.

8. QUALITY ASSURANCE PERSONNEL (COR)

TYPED NAME AND GRADE

SIGNATURE AND DATE

9. ISSUING AUTHORITY

TYPED NAME AND GRADE

SIGNATURE AND DATE

10. COR RESPONSE TO CONTRACTOR CORRECTIVE ACTION AND ACTION TAKEN TO PREVENT RECURRENCE

11. COR DETERMINATION

ACCEPTED REJECTED

12. CLOSE DATE

Appendix 3 - Customer Complaint Record

CUSTOMER COMPLAINT RECORD

DATE/TIME OF COMPLAINT

SOURCE OF COMPLAINT

ORGANIZATION

BUILDING NUMBER

INDIVIDUAL

PHONE NUMBER

NATURE OF COMPLAINT

CONTRACT REFERENCE

VALIDATION

DATE/TIME CONTRACTOR INFORMED OF COMPLAINT

ACTION TAKEN BY CONTRACTOR

RECEIVED/VALIDATED BY

Appendix 4 - Performance Assessment Report (PAR)

PERFORMANCE ASSESSMENT REPORT (PAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACT/TASK ORDER NUMBER 2. CONTRACTOR 3. TYPE OF SERVICES

4. QUALITY ASSURANCE PERSONNEL (COR) SIGNATURE AND DATE

5. COR PHONE 6. SUSPENSE DATE

I. PERFORMANCE

7. DEFICIENCY (CHECK ALL BOXES THAT APPLY)

NEW

REPEAT

NO DEFICIENCY NOTED

8. SERVICES SUMMARY or PWS PARAGRAPH ITEM REVIEWED

9. BRIEF DESCRIPTION OF DEFICIENCY (IF DEFICIENCY BOX WAS

CHECKED)

10. DETAILED PERFORMANCE ASSESSMENT

II. CONTRACTOR VALIDATION

11. CONTRACTOR REPRESENTATIVE CONCUR NON-

CONCUR

12. CORRECTIVE ACTION ESTIMATED COMPLETION DATE

13. CONTRACTOR REPRESENTATIVE CORRECTIVE ACTION AND PREVENTION OF RECURRENCE OR REASON FOR NON-

CONCURRENCE OF COR CITED DEFICIENCY

III. ACTION CORRECTED

14. CONCUR NON-CONCUR COR SIGNATURE AND DATE

15. COR REMARKS (REQUIRED)

6. CONTRACTOR REPRESENTATIVE REMARKS