Attachment_2_Draft_QASP_10.28.16.pdf

Quality Assurance Surveillance Plan (QASP)

For

MC2SA Software Maintenance Support and Software Technical/Development Support Contract # M67854-17-R-2602

1. PURPOSE.

This Quality Assurance Surveillance Plan (QASP) has been developed as the principal basis for assessing overall performance of the technical support for the MC2SA Software Maintenance Support and Technical/Development Support effort. This document will be used by the Government to evaluate the Contractor’s performance to ensure they are performing in a timely and effective manner, and are delivering the services and products specified in the contract.

This QASP provides the methodology by which the Contractor's performance will be monitored and establishes performance benchmarks that ensure a quantifiable basis for measuring effectiveness. The plan is designed to limit surveillance to the minimum necessary to verify that the Contractor is satisfactorily performing services directly related to the performance requirements of the Performance Work Statement (PWS). The performance requirements are those designated with corresponding performance measures and performance standards.

Surveillance will be accomplished in accordance with the Performance Requirements Summary

(PRS).

2. GOVERNMENT SURVEILLANCE

The Government will identify an individual to function as the Contracting Officer’s Representative (COR) for this requirement, and her/his authority will be limited to administering specific technical aspects of the contract. The COR will not provide direction that is outside the scope of responsibilities delineated under this contract. The designated individual will:

• Maintain a detailed knowledge of the technical requirements of the contract

• Document Contractor performance in accordance with the QASP

• Identify and immediately forward notifications of deficient or non-compliant performance to the Contracting Officer

• Approve priorities of support, resources, and associated schedules

3. SURVEILLANCE METHODS

Surveillance of Contractor performance is the method used by the COR to determine whether the Contractor is effectively and efficiently complying with all terms and conditions of the task order. In addition to statistical analysis, the functional expertise of the COR plays a critical role in adequately evaluating Contractor performance. The below listed methods of surveillance shall be used in the administration of this QASP and the standards are delineated in the Performance Requirements Survey (PRS) table below:

a. Inspection: A method carried out by visual examination of a product (e.g., meeting minutes, data from testing, plans, procedures, reports, or documentation) or process.

b. Analysis: A method that is carried out by detailed examination to assess the application of the techniques used in order to determine if they are appropriate and effective.

c. Demonstration: A method carried out by operation of a process from start to finish or the operation of the system, or part of the system, that relies on observable functional operation.

d. Customer Feedback: A method carried out by soliciting customer/end user satisfaction of technical services performed by the contractor.

4. PERFORMANCE REQUIREMENTS SUMMARY

The performance requirements requiring Government surveillance, the performance standards, and the surveillance methods are set forth in the PRS, below. The performance requirements correspond to paragraphs 4 and 5 of the PWS. The performance measures to be used are technical accuracy, clarity, completeness, timeliness, and effectiveness. The performance standards define what is considered acceptable performance and are the benchmarks against which performance will be measured. The Acceptable Quality Level (AQL) is the maximum acceptable variation from the performance standard. The surveillance method is how performance will be checked. The COR will use the PRS to evaluate the Contractor’s performance to assure they are timely, effective, and are delivering the services and products specified in the contract. Throughout contract performance, the PRS may be modified as necessary to ensure the satisfactory assessment of contract performance.

5. PERFORMANCE REVIEWS AND EVALUATIONS

a. COR Responsibility: The COR will conduct monthly performance reviews with the Contractor based upon the information obtained through the surveillance methods identified above pertaining to the performance outlined in the PRS. The following descriptors will be used for performance evaluations:

EXCELLENT: Performance significantly exceeds contractual requirements to the Government’s benefit; above the minimal acceptable level.

SATISFACTORY: Performance meets contractual requirements; minimal acceptable level.

UNSATISFACTORY: Performance does not meet contractual requirements; below minimal acceptable level.

b. Contractor Responsibility: The failure of the Contractor to meet the AQL set in the PRS may result in a reduction in contract payment. Reviews and evaluations will form the basis for determining compliance with performance requirements, measures, and standards. Performance reviews will be used to identify the degree of success or failure by the Contractor. Failure to perform due to Contractor delay or failure to comply with the mutually developed project schedule will be considered a failure in performance threatening the execution of the program and will result in a basis for Government consideration. The Government reserves the right to withhold payment of invoices until deficient action is completed and/or consideration is provided.

6. GOVERNMENT RESOURCES AND DEFINITIONS

The following definitions of Government Resources are applicable to this QASP:

a. Contracting Officer: A person duly appointed with the authority to enter into, administer, or terminate contracts and make related determinations and findings on behalf of the Government. The Contracting Officer is ultimately responsible for the final determination of the adequacy of the Contractor’s performance.

b. Contracting Officer’s Representative (COR): The COR is an individual designated and authorized in writing by the Contracting Officer to perform specific technical or administrative functions of a contract. The COR engages in Government surveillance of the Contractor’s performance. The COR may evaluate the Contractor’s performance and document any non-compliance, but only the Contracting Officer may take action against the Contractor for unacceptable performance. The COR is not authorized to make any commitments or changes that will affect price, quality, quantity, delivery, or any other term or condition of the contract.

Performance Requirement Summary

Performance Requirement

PWS

Para

Performance Measure

Performance Standard Acceptable Quality Level

(AQL)

Surveillance Method

Monthly Status Report

4.1.5, 4.1.12, 4.1.15

Technical Accuracy, Clarity, and Completeness

The report accurately addresses:

• All services provided and all events, products, and deliverables completed during the reporting period,

• All on-going activity/tasks for the upcoming reporting period, and

• Issues/concerns or problem areas, and actions planned or recommended solutions for alleviating the problem.

Report adheres to the established format.

Identifies 100% of the on-going activities/tasks.

Inspection

Timeliness The report is delivered on or before the 10th of each month.

Meets delivery schedule100% of the time (excluding Government caused delays).

Effectiveness Problems avoided or reduced once planned or recommended solutions implemented.

occurs with no required re-work at least 80% of the time.

Analysis and Customer Feedback

Integrated Product Team (IPT) and various Working

4.1.14, 4.2.4, 4.2.5, 4.2.6 4.4.3

Technical Accuracy, Clarity, and Completeness

Participates in various IPTs and working groups to provide technical and analytical support and produce technical reports that are technically accurate and understandable by the intended audience, tasks include:

The final version of each product adheres to the established format, is

Inspection and customer feedback

Requirement

PWS

Para

Performance Measure

Performance Standard Acceptable Quality Level

(AQL)

Surveillance Method

Group Participation

• Participate in weekly C2PC and/or TSOA technical review board (TRB) working groups.

• Verbally present any submitted Test Incident Reports (TIR).

• Provide C2PC and/or TSOA Subject Matter Expertise (SME) in the review of the respective TIRs.

• Conduct further analysis on any new/existing TIRs as directed by the Government.

• Technical Reports may be required for further engineering analysis as well.

• Participates in weekly C2PC and/or TSOA Test IPT.

• Provide C2PC and/or TSOA SME support in technical discussions.

• Provide in-depth technical status of ongoing test and engineering events.

• Participates in monthly JTCW/C2PC Team meetings.

• Provide in-depth technical status of ongoing test and engineering events as required.

complete (i.e., all constituent parts are present and each part is addressed in sufficient detail based on its intended use), technically accurate and incorporates Government review comments 100% of the time.

Timeliness Perform all tasks and produce products per established schedules.

• Required technical reports are due fifteen (15) days prior to test or engineering evaluation.

Final report is due five (5) days after

Meets schedule at least 100% of the time (excluding

Requirement

PWS

Para

Performance Measure

Performance Standard Acceptable Quality Level

(AQL)

Surveillance Method

Government review.

• Required TIRs are to be written and delivered to the Government within two (2) working days of systems error discovery, unless otherwise agreed upon by the Government.

Effectiveness Performance of tasks and products produced directly contribute to the successfully execution of the test event.

• Test Incident Reports identifies 90% of the mandatory information requirements such as detailed description of problem, system parameters, possible work-around and steps to re-create.

• Technical Reports contain 100% of relevant technical information to include description, analytical approach, conclusions and recommended courses of action.

required rework at least 90% of the time.

Inspection, Analysis, and Demonstration

Test Preparation

4.1.17, 4.1.19

Technical Accuracy, Clarity, and Completeness

Provide technical and analytical support and produce products that are technically accurate and understandable by the intended audience, tasks include:

• Prepare test plan, test cases/ scenarios, test procedures/scripts, and test data

• Setup and configure test environment and troubleshoot technical problems.

• Write or verify problem reports for technical accuracy and completeness.

The final version of each product adheres to the established format, is complete (i.e., all constituent parts are present and each part is addressed in

Inspection, Analysis, and Demonstration

Requirement

PWS

Para

Performance Measure

Performance Standard Acceptable Quality Level

(AQL)

Surveillance Method

Test plans adhere to the format and content requirements of the Government provided template, aligns with the documents listed in PWS paragraph section 5, and includes:

• Test case/scenario description and data flow.

• Traceability from each C2PC and/or TSOA requirement (or required C2PC and/or TSOA capabilities) covered by the test to the test case(s) that address it.

• Identification of allocated requirements not tested and the reason for not testing the requirement.

• Test procedures and individual test scripts include:

• Pre-test conditions requiring operator actions.

• Pre-test conditions that must be present in the test environment.

• Pre-test conditions for any datasets, files, or tools required for test execution.

• By step and subsystem, operator actions called out, along with pass/fail criteria based on requirement(s) or feature(s) tested.

sufficient detail based on its intended use), technically accurate, and incorporates Government review comments and are technically accurate 100% of the time.

Timeliness

Perform all tasks and produces products per established schedules.

• A draft test plan delivered to the Government

Meets schedule at least 90% of the time

Meets schedule at least 90% of the time

Requirement

PWS

Para

Performance Measure

Performance Standard Acceptable Quality Level

(AQL)

Surveillance Method fifteen (15) days prior to test or engineering evaluation. The final test plan delivered within seven (7) days after receipt of Government comments.

(excluding Government caused delays).

(excluding Government

Effectiveness

The level of detail for the products produced is based on the intended use and audience, and demonstrates a firm understanding of technical requirements and the test objectives.

Performance occurs with no required rework at least 80% of the time.

Analysis, Demonstration, and Customer Feedback

Test Execution

4.1.17, 4.1.19, 4.2.5

Technical Accuracy, Clarity, and Completeness

Provides technical and analytical support and produce products that are technically accurate and understandable by the intended audience, tasks include:

• Maintains test logs for all developmental test efforts.

• Write up TIRs as required

• Write or verify problem reports for technical accuracy and completeness.

required rework the time.

Analysis, Demonstration, and Customer Feedback

Performs all tasks and produces products per

Meets schedule the time (excluding Government

Inspection, Performance occurs with no

Requirement

PWS

Para

Performance Measure

Performance Standard Acceptable Quality Level

(AQL)

Surveillance Method demonstrates a firm understanding of technical requirements and the test objectives.

required rework at least 80% of the time.

and Demonstration

Test Analysis 4.1.19 Technical Accuracy, Clarity, and Completeness

Provides technical and analytical support and produce products that are technically accurate and understandable by the intended audience, tasks include:

• Conducts post-test analysis

• Reports on test results against all testable requirements defined in test plan.

• Identification of allocated requirements not tested and the reason for not testing the requirement.

• A draft test report is delivered per the established schedule. The final test report delivered within 10 days after receipt of Government comment.

• Test procedures and test scripts delivered per the established schedule.

Meets schedule the time (excluding demonstrates a firm understanding of technical requirements and the test objectives.

Performance occurs with no required rework at least 80% of

Inspection, Analysis, and

Requirement

PWS

Para

Performance Measure

Performance Standard Acceptable Quality Level

(AQL)

Surveillance Method the time.

External Test Technical Assistance and Analysis

4.1.19 Technical

Accuracy, Clarity, and Completeness

Provides technical and analytical support for external test events and produce products that are technically accurate and understandable by the intended audience, tasks include:

• Assist external test events as a C2PC and/or

TSOA subject matter expert (SME).

• Assist in troubleshooting technical problems with setup and configuration of the test environment.

• Verify technical accuracy and of TIRs.

Products adhere to the established format and are technically accurate 90% of the time.

Inspection, Analysis, Demonstration and feedback

• Technical assistance with test environment allows testing to start on time.

• Verification of TIRs are completed prior to test report being finalized

Meets schedule at least 90% of the time (excluding demonstrates a firm understanding of technical requirements and the test objectives.

• Performance of tasks and products produced directly contribute to the successfully execution of the test event.

Performance occurs with no required rework at least 80% of the time.

and feedback