ATTACH__7_-_QASP.docx

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP) FOR

THE JOINT TECHNICAL COORDINATING GROUP FOR MUNITIONS EFFECTIVENESS (JTCG/ME)

AND

THE U.S. ARMY MATERIEL SYSTEMS ANALYSIS ACTIVITY (AMSAA)

“PERFORMANCE OF ANALYTICAL SERVICES ASSOCIATED WITH TARGET DESCRIPTIONS, VULNERABLITY, SURVIVABILITY AND WEAPONS EFFECTIVENESS STUDIES, AND SUPPORT TO CURRENT OPERATIONS”

TASK ORDER CONTRACT

Prepared by:

Joint Technical Coordinating Group for Munitions Effectiveness (JTCG/ME) Aberdeen Proving Ground, MD 21005

TABLE OF CONTENTS

PAGE

1.0 INTRODUCTION

This Quality Assurance Surveillance Plan (QASP) has been developed to provide the overarching standard of surveillance for monitoring the Joint Technical Coordinating Group for Munitions Effectiveness (JTCG/ME) and U.S. Army Materiel Systems Analysis Activity (AMSAA), Analytical, Engineering, Technical and Management Support Services Task Order Contract and to provide a systematic approach for conducting the surveillance. This QASP describes how the Government personnel responsible for the JTCG/ME and AMSAA Task Order Contract will observe, evaluate and document the contractor’s performance in meeting the critical performance standards identified in the contract. Specifically, the QASP will identify the resources responsible for monitoring contractor performance; the monitoring methods that will be utilized in determining the quality level of services and products delivered under the contract; and the documentation requirements for this plan.

2.0 AUTHORITY

Authority for issuance of this QASP is provided under Contract Section E – Inspection and Acceptance, which provides for inspection and acceptance of the services and documentation called for in task orders (TOs,) to be executed by the Contracting Officer (CO) or a duly authorized representative.

3.0 SCOPE

The proposed contract which this QASP will cover, will be an Indefinite Delivery Indefinite Quantity (IDIQ) type contract for analytical, engineering, technical and management support services to assist JTCG/ME, AMSAA, and their customers in the conduct of operations research, systems and logistics analyses, studies and assessments to support Department of Defense objectives. The contracted support will be on a TO basis in the following general areas: (1) vulnerability analysis and assessment, (2) effectiveness analysis and performance assessments, (3) test and experiment conduct and support, and (4) support to current operations, publications, database, and reporting.

This document will serve as an overarching QASP for the contract with each individual TO having a QASP specifically tailored for the work specified in the individual TO Performance Work Statement (PWS). The individual TO QASP should be prepared as a joint effort between the customer and the Contracting Officer’s Representative (COR) and/or Task Order COR (TOCOR)/Government Technical Representative (GTR). The QASP is not a part of the contract nor is it intended to duplicate the Contractor’s Quality Control Plan (QCP). The Government may provide the Contractor an information copy of the QASP as an Attachment to each TO solicitation to support the Contractor’s efforts in developing a QCP and to inform the contractor as to how the Government intends to inspect performance on that specific TO.

4.0 GOVERNMENT RESOURCES AND RESPONSIBILITIES

To fully understand the roles and responsibilities of the parties administering this QASP it is important to first define the distinction in terminology between Quality Control Plan (QCP) and the QASP. The Contractor, and not the Government, is responsible for management and quality control actions necessary to meet the quality standards set forth by the contract. The Contractor develops and maintains a comprehensive QCP to ensure services are performed in accordance with the Performance Work Statement (PWS). The QASP on the other hand, is put in place to provide Government surveillance and oversight of the Contractor’s quality control efforts to assure that they are timely, effective, and are delivering the results specified in the contract. The following Government resources will have responsibility for the implementation of this QASP:

Contracting Officer (CO) – The CO ensures performance of all necessary actions for effective contracting and ensures compliance with the terms of the contract and safeguards the interests of the Government in the contractual relationship. It is the CO that assures the contractor receives impartial, fair and equitable treatment under the contract. The CO is ultimately responsible for the final determination of the adequacy of the Contractor’s performance.

Contracting Officer’s Representative (COR) - The COR is responsible for technical administration of the contract and assures proper Government surveillance of the Contractor’s performance. The COR is not empowered to make any contractual commitments or to authorize any contractual changes on the Government’s behalf. Any changes that the Contractor deems may affect contract, price, terms, or conditions shall be referred to the CO for action.

Task Order COR (TOCOR) and or Government Technical Represent (GTR) – The TOCOR and/or GTR provides detailed technical oversight of the Contractor’s performance and reports his or her findings to the COR in a timely, complete and impartial fashion to support the COR’s technical administration activities. While the TOCOR and/or GTR may serve as a direct conduit to provide Government guidance and feedback to the Contractor on technical matters, he or she is not empowered to make any contractual commitments or to authorize any contractual changes on the Government’s behalf. Any changes that the Contractor deems may affect contract, price, terms or conditions shall be referred to the CO for action.

5.0 METHODS OF SURVEILLANCE

Below is a description of the Performance Requirements Summary (PRS) as well as the methods of surveillance and the required documentation (i.e. PRS, Customer Feedback Form, and Monthly and Annual Surveillance Checklist) that will be used in the administration of this QASP. Surveillance methods and documents other than those identified in this section may be used; however, they must be fully documented and approved by the CO before implementation.

5. 1 Performance Requirements Summary. The PRS, at Attachment A provides an overall framework to which performance outcomes will be measured, along with incentives and/or disincentives under this contract. An individual TO QASP will contain a PRS that may include elements directly from the PRS at Attachment A, or may contain tailored elements to meet specific requirements in the individual TO PWS. For example, the contract level PRS calls for overall accuracy and completeness of deliverables, whereas a TO PRS may outline specific requirements for updating a particular model.

5.3 One Hundred-Percent Inspection. This method will be used for all contractor submissions, products and deliverables (tangible items). The COR, with assistance from the TOCOR/GTR and CO if necessary, accepts or rejects these products based on the requirements specified in the TO criteria and performance standards outlined in the PRS. Inspections will be documented monthly using the Monthly Surveillance Checklist at Attachment B which may be tailored to meet the specific requirements in each TO.

5.2 Customer Feedback. Customer feedback will be obtained monthly and documented using the Monthly Customer Feedback Form at Attachment B. Additionally, immediate customer feedback may be obtained from random customer complaints. Customer complaints, to be considered valid, must set forth clearly and in writing the detailed nature of the complaint; must be signed; and must be forwarded to the COR and CO. The COR shall maintain a summary log of all formally received customer complaints as well as a copy of each complaint in a documentation file.

5.4 Annual Surveillance Assessment. The COR and/or TOCO /GTR will perform an annual assessment of each contractor’s performance on TOs awarded during the annual report period. Unless specified otherwise by the CO, the annual report period will run for one year from contract award date and one year from the date of exercising an option thereafter. The Annual Surveillance Assessment will be documented on the Annual Surveillance Assessment Worksheet at Attachment D. If required, the Annual Surveillance Assessment Worksheet may be tailored to meet the requirements of an individual TO.

5.5 Surveillance Schedule. The COR and/or TOCOR/GTR will develop a surveillance schedule that is appropriate for each TO requirement. The Surveillance schedule should be included in the individual TO QASP.

6.0 Performance Standards and Acceptable Quality Level (AQLs)

6.1 Performances Standards. The QASP provides a systematic method to evaluate the service the Contractor is required to provide. The COR will assess the Contractor’s performance to ensure the Contractor is performing up to the specified standards. The performance standards for this PWS are stated in the PRS included in this QASP.

6.2 Acceptable Quality Level. The contract AQLs have been established in the PRS and provide performance standards that the Contractor shall meet in performing the contract. The COR and or TOCOR/ GTR will develop AQLs appropriate for each TO requirement. In the event the Contractor does not meet the established AQLs, the Contracting Officer has the right to exercise the negative incentives stated in the QASP.

6.3 Notification. The COR will notify the CO, in writing, of unacceptable quality levels. The CO will promptly provide written notification of discrepancies on a monthly basis, or immediately if the situation significantly affects the well-being of the project. The Contractor will be given the opportunity to respond, in writing, to each discrepancy.

6.4 Response. The Contractor shall provide a written response to the cause and the corrective action to prevent recurrence within ten (10) working days after receipt of Contracting Officer’s notification. The Contractor shall cite specific quality assurance program procedures or new procedures instituted to prevent recurrence.

6.5 Disincentives. The COR may evaluate the Contractor’s performance and document any non-compliance, but only the Contracting Officer may take action against the Contractor for an unacceptable rating. If any of the performance requirements do not meet the AQL set in the PRS table, the COR shall document the discrepancy(s) and shall notify the Contractor promptly, and the Contracting Officer, on a monthly basis, for appropriate action. When the performance is below the AQL standard, the Government may implement a negative incentive as specified in the contract/TO and or QASP. The COR will determine what action is appropriate for the specific circumstances and make a recommendation to the Contracting Officer.

7. 0 Records. The COR and TOCOR/GTR will maintain contract surveillance files as directed by the Contracting Officer. Surveillance files will include, in part, the following documentation for this contract and any subsequent TO awarded against this contract:

a. COR and TOCOR/GTR designation and appointment letters.

b. A copy of the contract and all contract modifications.

c. Copies of all Task Orders.

d. QASP and any other tools and checklists

e. All correspondence including Contracting Officer notifications and the Contractor’s responses.

f. Evidence of satisfactory performance provided by TOCOR/GTRs monthly.

g. Evidence of cost monitoring to include invoices certified and tracked against contract and/or task order ceilings.

h. Evidence that all materials and equipment obtained under a task order is accounted for and remains in Government possession.

i. Monthly and Annual Assessments. Out-of-cycle reports may be submitted in support of unacceptable performance identified during the performance period by the COR, or when there is a significant change in performance that alters the assessment in one or more evaluation areas.

j. Surveillance reports and a summary of the Surveillance Logs will be prepared by the COR, and submitted to the Contracting Officer annually.

8.0 QUALITY CONTROL PLAN

The contractor, not the Government, is responsible for management and quality control actions necessary to meet the quality standards for the products and services set forth by the contract and individual TOs issued. The quality control program is the driver for quality. The contractor shall develop and maintain a comprehensive

Quality Control Plan (QCP) to ensure services are performed in accordance with this PWS. The contractor’s quality control program shall verify and validate that the product development process is complete, in compliance with contract requirements, and meets customer expectations. The QCP shall be submitted as part of the contractor’s proposal and for any subsequent individual TOs written against this contract. Upon Contracting Officer approval, the QCP shall be incorporated into the contract. The contractor shall receive the Contracting Officer’s acceptance in writing prior to any proposed changes to its quality control program.

Once the quality control program is accepted, careful application of the process and standards presented in this document will ensure a robust quality assurance program. An annual review of the QCP will be conducted by a joint government and contractor team, with subsequent delivery to the government of any agreed upon changes to the Plan.

9.0 ATTACHMENTS

Attachment A - Performance Requirements Summary Attachment B - Monthly Surveillance Checklist Attachment C - Customer Feedback Form Attachment D - Annual Assessment Checklist

ATTACHEMENT A

Performance Requirements Summary for the JTCG/ME/AMSAA “Performance of analytical services associated with target descriptions, vulnerability, survivability and weapons effectiveness studies, and support to current operations”

TASK ORDER CONTRACT

and Acceptable Quality Level (AQL)

PRS #1 (PWS C.3.0)

Staffing stability and resource planning.

1. Management of personnel (retention, replacements, etc.) does not negatively impact quality or timeliness of products and services.

2. Correct allocation and proper mix of resources / skills / labor categories to complete PWS.

3. AQL: 95% of the time 100% inspection by COR,GTR and/or TOCOR

(-) Not award future orders unless there is remediation with the CO.

PRS #2 (PWS C.3.0)

Delivery of high quality deliverables.

1. Deliverables were delivered accurate, complete and incorporate all pertinent data with no (or minor) adjustments required by the Government.

2. Technical deliverables were developed and updated in accordance with the TO requirements.

3. Any errors identified by the Government are corrected within 5 business days.

4. AQL: 95% of the time 100% Inspection by COR, GTR and/or TOCOR

(-) Not award future orders unless there is remediation with the CO.

PRS #3 (PWS C.3.0)

Provided the required quantity of deliverables and services on time.

1. Services and deliverables were submitted according to, or ahead of, agreed upon schedule.

2. AQL: 95% of the time.

100% Inspection by COR, GTR and/or TOCOR

(-) Not award future orders unless there is remediation with the CO.

PRS #4 (PWS C.3.0)

Positive Business Relations, Customer Satisfaction and Timely and Open Communication.

1. The contractor was responsive to Government concerns and promptly applied corrective action to performance issues.

2. No validated customer complaint received for the reporting period.

3. The contractor provided timely notification of issues and/or items requiring Government actions.

4. AQL: 95% of the time 100% Customer Feedback

(-) Not award future orders unless there is remediation with the CO.

PRS #5 (PWS C.3.0)

Effective cost control.

1. Invoices demonstrate that the TO is being accomplished within the negotiated price for all items Exceptions documented and pre- approved by the Government.

2. AQL: 95% of the time.

100% Inspection by COR, GTR and/or TOCOR.

(-) Not award future orders unless there is remediation with the CO.

ATTACHMENT B

Monthly Surveillance Checklist JTCG/ME and AMSAA Task Order Contract Contract Number: Task Number: Report for Month of Year_

The TOCOR and/or GTR will perform a monthly assessment using the criteria established in the PRS. The table below is representative of the monthly surveillance checklist which can be tailored for each individual TO.

Description Surveillance Method/ Measure

Surveillance Dates

Rating

Comments Required if Rating Grade is Yellow or Red

PRS #1 (PWS C.3.0)

Staffing stability and resource planning 100% Inspection

PRS #2 (PWS C.3.0)

Delivery of high quality deliverables 100% Inspection

PRS # 3 (PWS C.3.0)

Provided the required quantity of deliverables and services on time 100% Inspection

PRS #5 (PWS C.3.0)

Effective cost control 100% Inspection

Rating Key

Yellow (Y) Met PRS criteria at least 95% but less than 100 % of the time; has two or more issues that were not minor, such as missed milestones, quality levels of documents requiring multiple reviews and rewrites, significant or serious complaints by customer(s) or documents that did not comply with tastings/deliverables set forth in the PWS.

Red (R) Has met the PRS criteria less than 95% of the time; performance has been at a level where the Contracting Officer has had to issue one or more cure notices regarding performance. Monthly performance that continues at the “Red” level will typically be at a level where a termination will be considered if the performance is not improved.

ATTACHEMNT C

Monthly Customer Feedback Form JTCG/ME and AMSAA Task Order Contract Contract Number: Task Number: Report for Month of Year_ Surveillance shall be completed monthly for each TO. The customer evaluation form may be tailored for individual TO requirements.

(AQL)

Surveillance Date

Rating Comments Required if Rating / Grade is Red

PRS #4 (PWS C.3.0)

Positive Business Relations, Customer Satisfaction and Timely and Open Communication

1. The contractor was responsive to Government concerns and promptly applied corrective action to performance issues.

2. No validated customer complaint was received for the reporting period.

3. The contractor provided timely notification of issues and/or items requiring Government actions.

4. AQL: 95% of the time

Rating Key

Red Has not met the PRS criteria; performance has been at a level where the Contracting Officer has had to issue one or more cure notices regarding performance. Monthly performance that continues at the “Red” level will typically be at a level where a termination will be considered if the performance is not improved.

ATTACHMENT D

Annual Surveillance Assessment Checklist for the

The COR and/or TOCOR/GTR will perform an annual assessment of each contractor’s performance for all TO issued to the contractor during the report period. The COR and/or TOCOR/GTR should complete this form utilizing the evaluation rating definitions listed below and the Task Order performance completion summary of ratings.

Quality of Performance

Review of Task Order Monthly Ratings.

TBD

Schedule

Cost Control

Business Relations

Management

Other Areas

Quality of Performance – Assess the contractor’s effort to transform operational needs and requirements into an integrated solution. Areas of focus may include the planning and management of program tasks, the quality of support provided throughout all phases of contract execution, the integration of program management specialties, management of interfaces, and the management of a totally integrated effort of all program management concerns to meet cost, performance, and schedule objectives. Assess how successfully the contractor meets program quality.

Schedule – Assess the contractor’s adherence to the required delivery schedule by assessing the contractor’s efforts during the assessment period that contribute to or effect the schedule variance. Also address significance of scheduled events (i.e., design reviews), discuss causes, and assess the effectiveness of contractor corrective actions.

Cost Control – Assess the contractor’s effectiveness in forecasting, managing, and controlling contract cost. Is the contractor experiencing cost growth or under run? If so, discuss the causes and contractor-proposed solutions for the cost overruns. For contracts where task or contract sizing is based upon contractor provided person-hour estimates, the relationship of these estimates to ultimate cost should be assessed. In addition, the extent to which the contractor demonstrates a sense of cost responsibility, through the efficient use of resources in each work effort should be assessed.

Business Relations – Assess the timelines, completeness and quality of problem identification, corrective action plans, proposal submittals, the contractor’s reasonable and cooperative behavior, effective business relations, and customer satisfaction.

Management – Assess the contractor’s success with timely award and management of subcontracts, including whether the contractor met small/small disadvantage and women-owned business participation goals. Discuss the extent to which the contractor discharges its responsibility for integration and coordination of all activity needed to execute the contract; identifies and applies resources required to meet schedule requirements; assigns responsibility for tasks/actions required by contract; communicates appropriate information to affected program elements in a timely manner. Assess the contractor’s risk mitigation plans. If applicable, identify any other management areas that are unique to the contract.

Other areas – Assess additional evaluation areas unique to the contract or that cannot be captured elsewhere.

Quality of Product or Service – Assess the contractor’s effort to transform operational needs and requirements into an integrated solution. Areas of focus may include the planning and management of program tasks, the quality of support provided throughout all phases of contract execution, the integration of program management specialties, management of interfaces, and the management of a totally integrated effort of all program management concerns to meet cost, performance, and schedule objectives. Assess how successfully the contractor meets program quality.

Schedule – Assess the contractor’s adherence to the required delivery schedule by assessing the contractor’s efforts during the assessment period that contribute to or effect the schedule variance. Also address significance of scheduled events (i.e., design reviews), discuss causes, and assess the effectiveness of contractor corrective actions.

Business Relations – Assess the timelines, completeness and quality of problem identification, corrective action plans, proposal submittals, the contractor’s reasonable and cooperative behavior, effective business relations, and customer satisfaction.

Management – Assess the contractor’s success with timely award and management of subcontracts, including whether the contractor met small/small disadvantage and women-owned business participation goals. Discuss the extent to which the contractor discharges its responsibility for integration and coordination of all activity needed to execute the contract; identifies and applies resources required to meet schedule requirements; assigns responsibility for tasks/actions required by contract; communicates appropriate information to affected program elements in a timely manner. Assess the contractor’s risk mitigation plans. If applicable, identify any other management areas that are unique to the contract.

Other areas – Assess additional evaluation areas unique to the contract or that cannot be captured elsewhere. The evaluation ratings are as follows:

Exceptional – Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the task and sub-task being assessed was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

Very Good - Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the task and sub-task being assessed was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

Satisfactory - Performance meets contractual requirements. The contractual performance of the task and sub-task contain some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

Marginal - Performance does not meet some contractual requirements. The contractual performance of the task and sub-task being assessed reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented.

Unsatisfactory – Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the task or sub-task contains a serious problem(s) for which the contractor’s corrective actions appear or were ineffective.