ATTACHMENT_J2_QASP.pdf

RFQ#:M68909-13-Q-7646

PWS Enclosure (1)

ATTACHMENT J2

Quality Assurance Surveillance Plan (QASP) for

MAGTF C4I Certification Program Support Services

1. PURPOSE.

This Quality Assurance Surveillance Plan (QASP) has been developed as the principal basis for assessing overall performance of the technical support for the MAGTF C4I System of Systems Technical Support (SOSTS) Services effort. This document will be used by the Government to evaluate the Contractor’s performance to assure they are timely, effective, and are delivering the services and products specified in the contract.

This QASP provides the methodology by which the Contractor's performance will be monitored and establishes performance benchmarks that ensure a quantifiable basis for measuring effectiveness. The plan is designed to limit surveillance to the minimum necessary to verify that the Contractor is satisfactorily performing services directly related to the performance requirements of the Performance Work Statement (PWS). The performance requirements are those designated with corresponding performance measures and performance standards. Surveillance will be accomplished in accordance with the Performance Requirements Summary (PRS).

2. GOVERNMENT SURVEILLANCE

The Contracting Officer’s Representative (COR) is identified in the PWS. COR authority will be limited to administering specific technical aspects of the contract. The PWS will identify specific guidelines and clearly detail the limits of the COR’s authority. The COR will:

a. Monitor Contractor performance in accordance with this QASP.

b. Coordinate priorities of support, resources, and associated schedules.

3. SURVEILLANCE METHODS

Surveillance of contractor performance is the method used by the Government to determine whether the contractor is effectively and efficiently complying with all terms and conditions of the contract. The COR may use the surveillance methods listed below in the administration of this QASP:

a. Inspection: A method carried out by visual examination of a product (e.g., meeting minutes, data from testing, plans, procedures, reports, or documentation) or process.

b. Analysis: A method that is carried out by detailed examination to assess the application of the techniques used in order to determine if they are appropriate and effective.

c. Demonstration: A method carried out by operation of a process from start to finish or the operation of the system, or part of the system, that relies on observable functional operation.

d. Customer Feedback: A method carried out by soliciting customer/end user satisfaction of technical services performed by the contractor.

4. PERFORMANCE REQUIREMENTS SUMMARY

PWS Enclosure (1)

ATTACHMENT J2

The performance requirements requiring Government surveillance, the performance standards, and the surveillance methods are set forth in the PRS, Attachment (1). The performance requirements correspond to paragraph 3 of the PWS. The performance measures to be used are technical accuracy, clarity, completeness, timeliness, and effectiveness. The performance standards define what is considered acceptable performance and are the benchmarks against which performance will be measured. The Acceptable Quality Level (AQL) is the maximum acceptable variation from the performance standard.

The surveillance method is how performance will be checked. The COR will use the PRS to evaluate the Contractor’s performance to assure they are timely, effective, and are delivering the services and products specified in the contract.

5. PERFORMANCE REVIEWS AND EVALUATIONS

COR Responsibility: The COR will conduct monthly performance reviews and evaluations with the Contractor based upon the information obtained through the surveillance methods identified above pertaining to the performance outlined in the PRS. The following criteria will be used for performance evaluations:

EXCELLENT: Performance that meets or exceeds 95% of acceptable quality level will be considered excellent.

SATISFACTORY: Performance meets contractual requirements; minimal acceptable level.

UNSATISFACTORY: Performance does not meet contractual requirements; below minimal acceptable level.

Contractor Responsibility: The failure of the Contractor to meet the AQL set in the PRS may: require the contractor to re-perform unsatisfactory services; result in a reduction in contract price; or result in termination of the contract. Reviews and evaluations will form the basis for determining compliance with performance requirements, measures, and standards. Performance reviews will be used to identify the degree of success or failure by the Contractor. Failure to perform due to Contractor delay or failure to comply with the mutually developed project schedule will be considered a failure in performance threatening the execution of the program. The Government reserves the right to withhold payment of invoices until deficient action is completed and/or consideration is provided.

6. GOVERNMENT RESOURCES AND DEFINITIONS

The following definitions of Government Resources are applicable to this QASP:

Contracting Officer: A person duly appointed with the authority to enter into, administer, or terminate contracts and make related determinations and findings on behalf of the Government. The Contracting Officer is ultimately responsible for the final determination of the adequacy of the Contractor’s performance.

Contracting Officer’s Representative (COR). The COR is an individual designated and authorized in writing by the Contracting Officer to perform specific technical or administrative functions of a contract.

The COR engages in Government surveillance of the Contractor’s performance. The COR may evaluate the Contractor’s performance and document any non-compliance, but only the Contracting Officer may take action against the Contractor for unacceptable performance. The COR is not authorized to make any commitments or changes that will affect price, quality, quantity, delivery, or any other term or condition of the contract.

Attachment (1) to PWS Enclosure (1)

Performance Requirement Summary

Performance Requirement

PWS

Para

Performance Measure

Performance Standard

Acceptable Quality Level

(AQL)

Surveillance Method

Monthly Status Report

4.1 Technical

Accuracy, Clarity, and Completeness

The report accurately addresses:

• All services provided and all events and deliverables completed during the reporting period.

• All on-going activities/tasks for the upcoming reporting period.

• All problem areas and actions planned or recommended solutions for alleviating the problem.

• Captures number of man hours spent per event, and labor categories.

• Adheres to MCTSSA templates.

Identifies 90% of the on-going activities/tasks.

Inspection

Timeliness The report delivered no later than the 10th of the month.

Meets delivery schedule 90% of the time (excluding government caused delays).

Effectiveness Problems avoided or reduced as a result of implementing the planned or recommended solutions.

Performance occurs with no required rework at least 90% of the time.

Analysis, Demonstration, and Customer Feedback

Technical and Engineering Support Documentation

3.1, 4.2, 4.4

Technical Accuracy, Clarity, Completeness, and Effectiveness

The Technical and Engineering reports and products produced shall be delivered to the COR in the latest MCTSSA provided and approved templates or approved Commercial Best Practices.

A draft deliverable is due to the COR per the established due dates.

The final deliverable is due five (5) working days after receipt of Government comments unless a different schedule is agreed to between the COR and contractor due to the scope of required modifications.

Technical and Engineering documents adhere to MCTSSA provided and approved templates or approved Commercial Best Practices and are delivered complete and on time 90% of the time (excluding Government caused delays).

Products produced are technically accurate 90% of the time.

Performance occurs with no required

Inspection, and Analysis

Performance Requirement Summary

Performance Requirement

PWS

Para

Performance Measure

Performance Standard

Acceptable Quality Level

(AQL)

Surveillance Method

Timeliness Performs all tasks in accordance with established schedules unless a different schedule is agreed to between the COR and contractor due to the scope of required modifications.

Redlines made to the technical or engineering products are corrected and returned to the COR within five

(5) working days of receipt of Government comments.

Meets schedule at least 90% of the time (excluding Government caused

Effectiveness Technical and engineering products produced are technically accurate, and demonstrates a firm understanding of MAGTF C4I System of System requirements.

Products produced are technically accurate 90% of the time.

Performance occurs with no required

Inspection and

Feedback

Test and Evaluation support

3.2, 3.2.1, 3.2.2, 3.2.3, 4.3.1, 4.3.2, 4.4

Technical Accuracy, Clarity, and Completeness

Test support provided result in technically accurate reports and present findings in a manner that is understandable to the intended audience.

Evaluation reports meet the specific requirements of the intended tests, and are produced and delivered to the COR in the MCTSSA provided and approved templates.

Test and Evaluation technical reports produced including: Test Plans, Test Cases/Scenarios, Test Process, Test Procedures, Test Scripts, Data Collection and Extraction, and Data Analysis, capture the intent of the test to be executed and are submitted to the COR in the MCTSSA provided and approved templates.

All test plans include a draft test plan due twenty (20) working days prior to the start of a test and a final test plan due five (5) working days after receipt of Government comments

Test and Evaluation technical reports produced adhere to MCTSSA provided and approved templates and are technically accurate 90% of the time.

Performance occurs with no required rework at least 90% of the time

Inspection, Performance Requirement Summary

Performance Requirement

PWS

Para

Performance Measure

Performance Standard

Acceptable Quality Level

(AQL)

Surveillance Method unless a different schedule is agreed to between the COR and contractor due to the scope of required modifications.

The test procedures and individual test scripts will be submitted to the COR in the MCTSSA provided and approved templates and shall include at a minimum:

Pre-test conditions requiring operator actions.

Pre-test conditions that must be present in the test environment.

Pre-test conditions for any datasets, files, or tools required for test execution.

By step and subsystem, operator actions called out, along with pass/fail criteria based on requirement(s) or feature(s) tested.

Test procedures and individual test scripts are due ten (10) working days prior to the start of a test or dry-runs depending upon the event schedule.

Modifications or redlines made to the test procedures during test execution are to be submitted to the COR five (5) working days following completion of the test.

The post-test analysis report shall include:

Known assumptions and constraints with the system, software, equipment, or tool(s) used in support of the post-test analysis.

Results of data verification, reduction, and analysis and a discussion of them, including a summary of the most important findings. If any data was excluded from the analysis due to collection errors, bias, or other cases, the rationale should be given.

Results of TIR, PCR, and PTR analysis. Include

Performance Requirement Summary

Performance Requirement

PWS

Para

Performance Measure

Performance Standard

Acceptable Quality Level

(AQL)

Surveillance Method recommendations on each TIR, PCR, or PTR assigned an operational impact rating in accordance with JITC guidance.

All Trouble Incident Reports

(TIR).

Evaluation results for any technical issue related to the SUT.

A draft post-test analysis report is due fifteen (15) working days after completion of the test. A final post-test analysis report is due five (5) working days after receipt of Government comments unless a different schedule is agreed to between the COR and contractor due to the scope of required modifications.

The final version of each technical report produced adheres to the MCTSSA provided and approved templates, is technically accurate and incorporates Government review comments.

Timeliness Performs all tasks in accordance with established schedules unless a different schedule is agreed to between the COR and contractor due to the scope of required modifications

Draft test plans are due 20 working days prior to the test.

Test procedures and test scripts (as required) are due 10 working days prior to test.

Modifications or redlines made to the test procedures during test execution are due 5 working days following completion of the test.

Meets schedule at least 90% of the time (excluding

Effectiveness Test reports produced are technically accurate, consistent with the MCTSSA provided and approved templates and demonstrate a firm

Performance occurs with no required rework at least 90% of the time.

Inspection and

Performance Requirement Summary

Performance Requirement

PWS

Para

Performance Measure

Performance Standard

Acceptable Quality Level

(AQL)

Surveillance Method understanding of MAGTF C4I System of Systems test and evaluation technical requirements.

Performs of tasks and products produced directly contribute to the successfully execution of the test event.

Document Reviews

3.3, 4.4

Technical Accuracy, Clarity, Completeness , and Effectiveness

Results of each document reviewed is documented in a Technical Report that:

• Accurately describes technical discrepancies, errors, or omissions found in the document, including a reference to the appropriate paragraph, topic and substantiations.

• Propose changes/solutions that are feasible and technically accurate.

• Present findings in a manner that is understandable to the intended audience.

The final version of each technical report adheres to the MCTSSA provided and approved templates, is complete (i.e., all constituent parts are present and each part is addressed in sufficient detail based on its intended use), and incorporates Government review comments.

Documentation adheres to the MCTSSA provided and approved templates, and is delivered complete and on time 90% of the time (excluding Government caused delays).

Result of each document reviewed is technically accurate 90% of the time.

Performance occurs with no required

Inspection, Timeliness Performs all tasks in accordance with established schedules unless a different schedule is agreed to between the COR and contractor due to the scope of required modifications

Technical reports for documentation reviewed are delivered to the COR ten (10) working days after document receipt.

Meets schedule at least 90% of the time (excluding

Effectiveness Technical reports produced are technically accurate and demonstrates a firm understanding of MAGTF C4I System of Systems.

Performance occurs with no required rework at least 90% of the time.

Inspection and