Att 8 - QASP 05 Feb.doc

Quality Assurance Surveillance Plan (QASP)

For

United States Army Test and Evaluation Command Information Technology Support Services

1 Introduction

1.1 Mission

2 Purpose 3 Authority 4 Roles and Responsibilities

4.1 Contracting Officer (KO)

4.2 Contracting Officer's Representative (COR)

4.3 Technical Monitors

4.4 Contractor Representatives

5 Performance Requirements and Method of Surveillance

5.1 Quality Control Program

5.2 Government Quality Assurance……………………………………………………………….6

6 Identification of Required Performance Standards/Quality Levels.…………………………….6 7 Government Surveillance……………………………………………………………………….6

7.1 Primary Methods of Surveillance……………………………………………………………..6

7.2 Evaluation Methods……………..…………………………………………………………….7

7.3 Monthly Surveillance Schedule...…………………………………………………...………...8

7.4 Inspection Instruction/Checklists…………………………………………………………..….8

7.5 Documentation of Inspections….………………………………………………………

8 Identification of Required Performance Standards/Quality Levels.………………….…………9

8.1 Primary Methods of Surveillance…………………………………………………………..…9

8.2 Primary Methods of Surveillance…………………………………………………………..…9 9 Acceptance of Service..……………………………………………..…………..………….…..10 10 Monthly COR Activity Report…………………………………………….….……….… 11 Contractor Manpower Reporting (CMR)…………………………………..……….……..….10 12 Quality Assurance Surveillance File...……….…………………………………….…………10 13 Contractor Performance Assessment Reporting System (CPARS)…………………………..11 14 QASP Changes………………………………………………………………...…….… Appendix 1 - Standards and Acceptable Quality Level spreadsheet…………………………….12 1 Introduction Army Test and Evaluation Command (ATEC) is one of Department of Defense’s (DOD) foremost test and training organizations supporting DOD, non-DOD, and commercial materiel developers by testing a broad spectrum of equipment throughout their life cycles, from concept through deployment. As such, ATEC is not a material, item, or system developer. ATEC provides crucial test support to such developers who are ATEC's primary customers. This contract support is devoted primarily to support ATEC Headquarters (HQ) and its subordinate organizations as they conduct tests, operate and maintain the ranges, facilities, and equipment necessary to perform testing for their customers.

1.1 Mission

ATEC’s mission is to plan, conduct, and integrate developmental testing, independent operational testing, independent evaluations, assessments, and experiments in order to provide essential information to decision makers for our Nation's Warfighter. ATEC provides the most flexible, responsive, innovative, and diverse set of test capabilities, assets, and services throughout the Army for providing test and test support services for authorized customers.

To accomplish this, ATEC will:

a. Deliver value-added products and services to its customers;

b. Identify and optimize the process and capabilities essential to providing quality and timely products and services;

c. Provide, develop, sustain, and integrate people, technological capabilities, and procedures; and,

d. Identify, secure, and allocate resources necessary to meet ATEC’s mission requirement through efficient, flexible and responsive management.

To successfully support ATEC's mission, ATEC requires the ability to rapidly move, store, and retrieve data from a wide variety of sources and locations in a highly reliable and secure manner. ATEC requires a high level of technical expertise, flexibility, and depth of professional and technical expertise in order to meet its mission goals and objectives. A technical staff capable of modifying and operating sophisticated information management systems in a timely and cost-efficient manner is required to meet the needs of ATEC.

2. Purpose

The purpose of the Quality Assurance Surveillance Plan (QASP) is to describe the systematic methods the Government will use to evaluate Contractor actions while performing the requirements as laid out in the Performance Work Statement (PWS) for the AEC Support Services (AECSS). The QASP provides a means for evaluating whether the Contractor is meeting the performance standards and acceptable quality levels identified in the PWS and identification of surveillance methods used to document the performance.

The QASP has been developed by ATEC to provide direction to personnel performing contract surveillance activities. Personnel performing surveillance to the contract terms and conditions and PWS requirements will periodically review the QASP throughout the life of the contract. The QASP establishes procedures on how this assessment/inspection process will be conducted. It provides details for a continuous oversight process as follows:

· What will be monitored

· How monitoring will take place

· Who will be conduct the monitoring

· How monitoring efforts and results will be documented

The contractor is responsible for implementing and delivering performance that meets contract standards using its Quality Control Plan. The QASP provides the structure for the government's surveillance of the contractor's performance to assure that it meets contract standards. It is the government's responsibility to be objective, fair and consistent in evaluating contractor performance.

The QASP is not part of the contract nor is it intended to duplicate the contractor's quality control plan. This QASP is a living document. Flexibility in the QASP is required to allow for an increase or decrease in the level of surveillance necessary based on contractor performance. The government may provide a copy of the QASP to the contractor to facilitate open communication. In addition, the QASP should recognize that unforeseen or uncontrollable circumstances might occur that are outside the control of the contractor. As a result, the QASP should ensure early identification and resolution of performance issues to minimize impact on mission performance.

3 Authority

Authority for issuance of this QASP is provided under Part 46 of the Federal Acquisition Regulation, Inspection of Services clauses, which provides for inspection, acceptance and documentation of the service called for in the contract or order. This acceptance is to be executed by the Contracting Officer or a duly authorized representative.

4 Roles and Responsibilities

The following personnel shall oversee and coordinate surveillance activities:

4.1 Contracting Officer (KO)

The KO shall ensure performance of all necessary actions for effective contracting and compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The KO shall also ensure the Contractor receives impartial, fair, and equitable treatment under this contract. The KO shall determine and document the final assessment of the Contractor's performance in the Government’s past performance tracking system.

Assigned KO: Tara Casey Organization or Agency: US Army Contracting Command – Aberdeen Proving Ground Telephone: (410) 278-0863 Email: tara.l.casey.civ@mail.mil

4.2 Contracting Officer's Representative (COR)

The COR is responsible for providing continuous technical oversight of the Contractor's performance. The COR uses the QASP to conduct the oversight/surveillance process. The COR shall keep a Quality Assurance file that accurately documents the Contractor's actual performance. The purpose is to ensure the Contractor meets the performance standards contained in the contract. The COR is responsible for reporting early identification of performance problems to the KO. The COR is required to provide an annual performance assessment to the KO which will be used in documenting past performance. The QASP is the primary tool for documenting Contractor performance. The COR is not empowered to make any contractual commitments or to authorize any contractual change on the Government's behalf.

Assigned COR: Kristen Gatch Organization or Agency: U.S. Army Test and Evaluation Command

Telephone: 443-861-9782 Email: Kristen.m.gatch.civ@mail.mil

4.3 Technical Monitor(s)

A representative from the requiring activity that provides detailed technical oversight of the Contractor’s performance and reports his or her findings to the COR in a timely, complete, and impartial fashion to support the COR’s administration activities. Any changes that the Contractor deems may affect contract, price, terms, or conditions shall be referred to the Contracting Officer for action. Technical monitors may include performance monitors, inspectors, technical experts, or others who provide information that helps the COR monitor Contractor performance.

4.4 Contractor Representatives

The following employees of the Contractor serve as the Contractor's Program Manager and Task Manager for this contract. (To be completed after contract award)

Program Manager: <upon award, enter name>

Telephone: <enter number>

Email: <enter address>

Task Manager - <upon award, enter name>

Telephone: <enter number>

Email: <enter address>

Other Key Contract Personnel: <upon award, enter name or delete these lines if not applicable>

Title: <enter title>

Telephone: <enter number>

Email: <enter address>

5 Performance Requirements and Method of Surveillance

This section describes the special requirements for this effort. The following sub-sections provide details of various considerations on this effort.

5.1. Quality Control Program. The Contractor's Quality Control Program requirements are defined in paragraph C.1.18 of the PWS. The Contractor shall develop, maintain, enforce and document a Quality Control Plan (QCP). This plan shall comply with the requirements of this contract and outline how the Contractor will ensure the Government will receive all of the services required by the contract at the specified quality level.

5.2. Government Quality Assurance. The Government will perform Quality Assurance oversight of the Contractor and Contractor’s QCP IAW with this QASP. The Government will ensure acceptable levels of contract performance are achieved and that the Contractor is performing acceptable levels of Quality Control to ensure fully acceptable services are provided.

6. IDENTIFICATION OF REQUIRED PERFORMANCE STANDARDS/QUALITY LEVELS

The required performance standards and quality levels are included in the PWS in Attachment 4, “Standards and Acceptable Quality Level” spreadsheet.

7. GOVERNMENT SURVEILLANCE

In an effort to minimize the performance management burden, simplified surveillance methods will be used by the Government to evaluate Contractor performance. Surveillance will be performed in support of the following work requirements:

a.

Customer Support

b. Cybersecurity Operations

c. Configuration Management d.

General IT Support Operations

e. Software Development

f. Software Support Activity g.

Systems Administration Support

h. Logistics i.

Other Support Services

7.1 Primary Methods of Surveillance:

a. Customer Input – Formal Customer surveys will be generated and results reviewed. Feedback may be obtained from random customer complaints. Both methods will be used for items 6a, 6h, and 6i. Customer complaints, to be valid, must clearly outline in writing the detailed nature of the complaint, must be signed, and must be forwarded to the COR. The COR must maintain a summary log of all formally received customer complaints as well as a copy of each compliant in a documentation file. The COR must also keep the tabulated results of all customer satisfaction surveys on file and must maintain a summary log of the results.

b. Contractor Self-Reporting – Monthly status reports will be generated by the contractor and reviewed by the government for items 6a, 6b, 6c, 6d, 6e, 6f, and 6h. The reports are identified as deliverables in CDRLs A001 through A013:

A001 Contractor Access/Property Control Plan

A002 Quality Control Plan A003 Hazardous Material Report

A004 Contractor Gov’t Furnished Property Inventory Report

A005 Contractor Loss/Damage Report A006 Maintenance/Warranty Report

A007 Critical Service Outage A008 Reserved

A009 Software Documentation

A010 Cybersecurity Compliance Report

A011 Sensitive Items Inventory Report

A012 Contractor Progress Status and Management Report

c. The Contractor shall implement a system to manage (control, use, preserve, protect, repair and maintain) Government property in its possession. Reference PWS sections C.1.28 and C.3 for the above item 6h (Logistics). The Contractor shall initiate and maintain the processes, systems, procedures, records, and methodologies necessary for effective control of Government property. These processes shall be consistent with Government property management industry standards and/or leading practices except where prohibited by law or in conflict with regulatory guidance. During the period of performance, the Contractor shall disclose any significant changes to their property management system to the Government prior to implementation.

d. A complete inventory of all Secret/TS/SCI related items shall be accomplished as part of the phase-in/phase-out.

e. A complete inventory of all Government Furnished Property will be accomplished as part of the phase-in/phase-out.

f.

Random Sampling – Random inventory inspections will be performed for the above item 6g. No more than 4 inspections will be performed on approximately 10% of the inventory per year.

g. Periodic Observation - The Government will periodically monitor the Contractor’s performance for the above items 6b, 6c, 6g, and 6h.

7.2 Evaluation Method: The COR will review contractor status reports, evaluate customer surveys, and conduct contractor performance verification and validations. The COR and/or technical POC will observe Contractor performance and document findings. The COR will confirm and record the dates and times of performance problems and resolutions.

If the results determine that the number of defects exceed the Acceptable Quality Level (AQL), the month’s services will be considered unsatisfactory for the particular requirement under surveillance. If Government action or lack of action caused the unsatisfactory performance, the defects will not be counted. A memorandum will be prepared by the COR explaining the Government’s action and submitted with the monthly report to the KO. Unsatisfactory performance will be reported on DA Form 5479-R “Contract Discrepancy Report” (CDR). This form is used to notify the contractor of discrepancies found by the COR. This is where the contractor is allowed to answer how the discrepancy will be corrected and how reoccurrence will be avoided. If unsatisfactory performance occurs in an area based on the COR’s sampling, the contractor will be given a CDR. This completed and signed CDR and supporting documentation will be forwarded to the KO for disposition. A copy will be maintained in the COR file.

The COR will draft a Contracting Performance Assessment Report (CPAR) and submit to the KO for review. The contractor’s performance will be assessed and evaluated. The results will be entered into the Contracting Performance Assessment Reporting System (CPARS). CPARS will be accomplished annually or at the end of the Task Order whichever comes first.

7.3. Monthly Surveillance Schedule. The COR/Technical POC’s will develop a monthly schedule for surveillance activities based on the Performance Standards as outlined in the Standards and Acceptable Quality Level spreadsheet (SAQL). The SAQL is an attachment to this QASP as well as any other critical contractual requirements determined to be higher risk requiring oversight. The surveillance outlined in the monthly schedule will be as detailed and in-depth as necessary to provide the Government with the objective quality evidence required to support acceptance of the services provided by the Contractor. The schedule will provide planned periodic surveillance during all required hours of the Contractor’s operation. The schedule is “FOR OFFICIAL USE ONLY” and is not releasable to anyone other than authorized Government personnel. The schedule will identify the method of surveillance, the date of inspection, place of inspection, activity to be monitored, and who will conduct the surveillance (dates and times can be altered without a formal change to the schedule as long as all inspections are performed).

7.4. Inspection Instructions/Checklists. The COR will develop inspection instructions/checklists for all planned inspections which may require special instructions on how to perform the inspection, what observations should be made with specific details of what to look for in acceptable performance. The Surveillance Activity Checklists, DA Form 5476 http://armypubs.army.mil/eforms/pdf/A5476.PDF may be tailored to be used for this purpose.

7.5. Documentation of Inspections. All inspections/observations will be documented and will include at a minimum date and time, who performed the inspection, what was inspected, to what standard and the results. Successful or exceptional performance will be documented as well as deficiencies in Contractor performance. Documentation will be maintained for future reference, audit, and proof of inspection and past performance documentation.

8. Non-Conforming Performance. The COR will notify the Contractor, in person, each time an unacceptable observation has been recorded and ask the Contractor to correct the problem or re- accomplish the service. All valid non-conformances observed by the Government will be documented! The COR will record the contract requirement, the specific deficiency to the requirement, the date and time it was discovered, and have the Contractor initial the entry. If the work or support is not completed as specified, according to standards, is incorrect or is unsatisfactory, the Contractor will develop a cure plan to avoid further substandard work and re-perform the work at no additional cost or fee to the Government. The Contractor will be given a time line after notification to correct the deficiency in accordance with requirements of the contract. Deficiencies that cannot be corrected within the identified time line will be reported to the KO.

8.1. Customer Feedback. For customer feedback or complaints it is recommended that the Customer Complaint Record DA Form 5477 http://armypubs.army.mil/eforms/pdf/A5477.PDF be used. However, customer feedback can be in any form, verbal or written but no matter the format the COR will maintain a record of the feedback, positive or negative.

a. If immediate response to a complaint is required, the COR will provide the response to the requester as soon as practical within 24 hours.

b. The COR will conduct an investigation to determine the validity of any negative comments received.

c. If the negative comment is determined not to be valid, if possible, the COR will inform the customer of the reason(s) as soon as practical.

d. For validated negative comments that are true contractual non-conformances see paragraph 6. If this non-conformance is a more serious (systemic in nature) deficiency or recurring deficiency that could indicate a trend, the COR shall complete DA Form 5479-R (see paragraph 6.2.)

e. The COR will follow up with the customer of the corrective action taken by the Contractor, if applicable.

8.2. Contract Deficiency Reports. The COR will report more serious (systemic in nature) deficiencies or recurring deficiencies that could indicate a trend by using the DA Form 5479-R, Contract Discrepancy Report (CDR) http://armypubs.army.mil/eforms/pdf/A5479.PDF . The CDR will state the Contract Requirement, the specific contract reference and the specific violation (s) to the requirement. The COR will forward copies of CDRs to the KO within one working day. The KO will notify the Contractor and request corrective action to the specific non-conformance, identification of the root cause and corrective action for the root cause with a required suspense date to be determined on a case-by-case basis.

The KO may require the Contractor to re-perform any services that do not meet contract requirements at no additional cost to the Government. The Contractor shall return a copy of the completed and signed DA Form 5479-R along with their Corrective Action Plan (CAP) to the KO for review, approval and inclusion in the contract/order file. The KO will include the COR in the review of the CAP and re-inspection of services that were found deficient. The KO and COR will review the CAP to ensure it included adequate root cause analysis, corrective action for the root cause and that it was accomplished in the timeframe agreed upon. The KO and COR will not consider the CDR closed until all deficiencies have been corrected and all agreed upon actions of the CAP have been completed. Contractor’s failure to complete corrective actions will be reported immediately to the KO for further action. (Note: if the COR has to perform repeated re-inspections there could be a Government re-inspection cost which could be recovered from the Contractor). Post the completion of the Contractor’s corrective action, the COR will ensure that the Contractor’s corrective action was sufficient to preclude recurrence.

9. Acceptance of Services. When all services have been deemed acceptable and there is documented objective quality evidence to support acceptable performance the COR will accept the services provided and authorize payment upon satisfactory completion of the work. This is accomplished by approving the Contractor’s invoice in Wide Area Workflow https://wawf.eb.mil/.

10. Monthly COR Activity Report. The COR will submit a brief monthly activity report to the KO which will include the next month’s surveillance schedule, summary of inspections performed with results, summary of customer complaints with analysis, summary of contract discrepancies, results of data analysis, recommended adjustments to surveillance, invoices accepted and any other pertinent contract activities.

11. Contractor Manpower Reporting (CMR). The COR will complete required information and verify Contactor has complied with the requisite Contractor manpower reporting and document any observations. Reporting is for the 12 month period ending on 30 September of each FY and reporting is due by the Contractor by 31 December of each year. Registration and manual can be found at sam.gov

12. Quality Assurance Surveillance File: The official COR Quality Assurance Surveillance File will be maintained in the on-line COR Management Tool https://arc.army.mil/COR per ACC PAM 70-1. The Quality Assurance, or COR Surveillance file shall contain the following as applicable:

a. Copy of the COR designation letter from the KO, any changes to that letter, and any termination letters.

b. Training Certifications for COR and refresher training.

c. Copy of the contract and all contract modifications.

d. Copy of the applicable Quality Assurance Surveillance Plan (QASP). e. Copy of the Contractor’s Quality Control Plan (QCP)

e. All correspondence initiated by authorized representatives (Contractor or Government) concerning performance of the contract.

f. Names, position titles and contact information of all key personnel assigned to this contract both Government and Contractor.

g. Monthly surveillance schedules.

h. Surveillance Checklists

i. Records of all inspections performed and the results.

j. Customer Feedback.

k. Memoranda for record of minutes of any meeting, telephone conversations and discussions with the Contractor or others pertaining to the contract or contract performance.

l. Documentation pertaining to acceptance of services, reports or data

13. Contractor Performance Assessment Reporting System (CPARS). Documented surveillance data IAW this QASP will be used as objective data to support CPARS ratings. The inclusion of detailed documented inspection results provides objectivity to CPARS reporting. CPARS will be accomplished annually or at the end of a Task Order, whichever occurs first. http://www.cpars.csd.disa.mil/cparsmain.htm

14. QASP Changes. The QASP is a living document and, as such, may be changed as needed. However, the KO must approve changes. The COR will submit recommended changes to the KO for approval.

Appendix 1 - Standards and Acceptable Quality Levels spreadsheet