Attachment 1 QASP.pdf

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

U.S. Army Aberdeen Test Center (ATC) Safety Support Services

ATC Safety & Occupational Health Office 6943 Colleran Road

TEAT-COS

Aberdeen Proving Ground, MD 21005-5059

Issued 20 February 2020

TABLE OF CONTENTS

1 INTRODUCTION

1.1 Purpose

1.2 Performance Management Strategy

2 ROLES AND RESPONSIBILITIES

2.1 The Contracting Officer (KO)/ Contracting Officer’s Representative (COR)

2.2 The Government Technical Monitor (TM)

3 IDENTIFICATION OF REQUIRED PERFORMANCE STANDARDS/QUALITY

LEVELS

4 METHODOLOGIES TO MONITOR PERFORMANCE

4.1 Surveillance Techniques

4.2 Customer Feedback

5 QUALITY ASSURANCE DOCUMENTATION

5.1 The Performance Management Feedback Loop

5.2 Monitoring Form

6 ANALYSIS OF QUALITY ASSURANCE ASSESSMENT

6.1 Determining Performance

6.2 Reporting

6.3 Reviews and Resolution

ATTACHMENT 1: PERFORMANCE REQUIREMENTS SUMMARY

ATTACHMENT 2: SAMPLE QUALITY ASSURANCE MONITORING FORM

ATTACHMENT 3: SURVEILLANCE ACTIVITY CHECKLIST

QUALITY ASSURANCE SURVEILLANCE PLAN

(QASP)

1 INTRODUCTION

This quality assurance surveillance plan (QASP) is pursuant to the requirements listed in the Performance Work Statement (PWS) entitled ATC Safety Support Services. This plan sets forth the procedures and guidelines U.S. Army Aberdeen Test Center will use in ensuring the required performance standards or services levels are achieved by the contractor.

1.1 Purpose

The purpose of the QASP is to assure that the performance of specific activities and the completion of project milestones are accomplished in accordance with all requirements set forth in the PWS and outlined in the Project Management Plan (PMP). This QASP describes the mechanism for documenting noteworthy accomplishments or discrepancies for work performed by the contractor. Information generated from the Technical Monitor’s (TM) surveillance activities will directly feed into performance discussions with the contractor by the Contracting Officer. The intent is to ensure that the contractor performs in accordance with performance metrics set forth in the contract documents, the Government receives the quality of products and services called for in the contract, and the Government only pays for acceptable products and services received.

The QASP is intended to accomplish the following:

Describe the roles and responsibilities of participating Government officials.

Identify the key milestones, deliverables, and standards that will be assessed.

Describe the surveillance methodology that will be employed by the Government in assessing the contractor’s performance.

Describe the surveillance documentation process and provide copies of the forms that the Government will use in evaluating the contractor’s performance.

Define Exceptional, Very Good, Satisfactory, Marginal, and Unsatisfactory performance standards for key milestones, deliverables, and standards.

Outline corrective action procedures.

1.2 Performance Management Strategy

The contractor is responsible for the quality of all work performed. The contractor measures that quality through the contractor’s own quality control (QC) program. QC is work output, not workers, and therefore includes all work performed under this contract regardless of whether the work is performed by contractor employees or by subcontractors.

2 ROLES AND RESPONSIBILITIES

2.1 The Contracting Officer (KO)/ Contracting Officer’s Representative (COR) The KO is responsible for monitoring contract compliance, contract administration, and cost control and for resolving any differences between the observations documented by the COR and the contractor. The KO will designate a COR as the government authority for performance management. The number of additional representatives serving as technical monitors depends on the complexity of the products/ services measured, as well as the contractor’s performance, and must be identified and designated by the KO.

2.2 The Government Technical Monitor (TM)

The KO may appoint a Government TM to assist the COR in monitoring a contractor's performance. The KO may appoint a TM because of physical proximity to the contractor's work site, or because of special skills or knowledge necessary for monitoring the contractor's work. The KO may also appoint a TM to represent the interests of another requirements office or post concerned with the contractor's work. A TM shall be a direct-hire U.S. Government employee.

The role of the contract TM involves numerous techniques and procedures for determining whether satisfactory delivery and contract completion is likely. A contract is monitored to determine whether it will be performed in accordance with the contract requirements and address problems as they occur. The TM should refer all contracting issues to the KO.

The TM is responsible for technical administration of the project and ensures proper government surveillance of the contractor’s performance. The TM is not empowered to make any contractual commitments or to authorize any contractual changes on the government’s behalf. Any changes that the contractor deems may affect contract price, terms, or conditions shall be referred to the KO for action. The TM will have the responsibility for completing QA monitoring forms used to document the inspection and evaluation of the contractor’s work performance. Government surveillance may occur under the inspection of services clause for any service relating to the contract.

3 IDENTIFICATION OF REQUIRED PERFORMANCE STANDARDS/QUALITY

LEVELS

The required performance standards and/or quality levels are listed below and shall be included in Attachment 1, “Performance Requirements Summary.”

Quality (of Product or Service) – Does the product or service provided meet the specifications of the contract? Are the reports/data accurate? Does the contractor’s work measure up to commonly accepted technical or professional standards?

Schedule – Assess the timeliness of the contractor against the completion of the contract, milestones, delivery schedules, and administrative requirements. Include the contractor’s efforts that contribute to or affect the schedule variance (positively or adversely).

Management of Key Personnel – Assess the performance of the contractor’s key personnel. If a key person did not perform well, what action was taken by the contractor to correct this?

Cost Control – Assess the contractor’s effectiveness in forecasting, managing, and controlling contract cost. Does the contractor keep within the total estimated cost?

Did the contractor do anything innovative that resulted in cost savings? The extent to which the contractor demonstrates a sense of cost responsibility, through the efficient use of resources, in each work effort should be assessed.

Business Relations – Is the customer satisfied? Is interaction/communication between the contractor and the government satisfactory? Address the timeliness of awards to subcontractors and management of subcontractors. Identify the quality of problem identification and corrective actions. Include the contractor’s history of reasonable and cooperative behavior.

Utilization of Small Business – Document whether the contractor is utilizing a Small Business, (including Small Disadvantaged Businesses) to participate in contract performance consistent with efficient performance of the contract.

Other Areas – Specify additional evaluation areas that are unique to the contract or that cannot be captured elsewhere on the form.

4 METHODOLOGIES TO MONITOR PERFORMANCE

4.1 Surveillance Techniques

In an effort to minimize the performance management burden, a simplified surveillance method shall be used by the government to evaluate contractor performance. The method of surveillance is 100% Inspection – Each month, the TM shall review the generated documentation and assess per the Surveillance Activity Checklist (in Attachment 3) and enter summary results into the Quality Assurance Monitoring Form (in Attachment 2).

4.2 Customer Feedback

The contractor is expected to establish and maintain professional communication between its employees and customers. The primary objective of this communication is customer satisfaction. Customer satisfaction is the most significant external indicator of the success and effectiveness of all services provided and can be measured through customer complaints.

Performance management drives the contractor to be customer focused through initially and internally addressing customer complaints and investigating the issues and/or problems but the customer always has the option to communicate complaints to the COR as opposed to the contractor.

Customer complaints, to be considered valid, must set forth clearly and in writing the detailed nature of the complaint, must be signed, and must be forwarded to the COR. The COR will accept those customer complaints and investigate using the Quality Assurance Monitoring Form, identified in Attachment 2.

Customer feedback may also be obtained from the results of formal customer satisfaction surveys.

5 QUALITY ASSURANCE DOCUMENTATION

5.1 The Performance Management Feedback Loop

The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed in the PWS and are assessed using the acceptable quality levels (AQLs) for contractor performance as defined in Attachment 1.

5.2 Monitoring Form

The government’s QA surveillance, accomplished by the TM/COR, will be reported using the monitoring form in Attachment 2. The form, when completed, will document the government’s assessment of the contractor’s performance under the contract to ensure that the required results are being achieved.

5.2.1 The NDCEE Program Management Office (PMO) will retain a copy of all completed QA surveillance forms.

6 ANALYSIS OF QUALITY ASSURANCE ASSESSMENT

6.1 Determining Performance

6.1.1 Government shall use the monitoring methods cited to determine whether the performance standards/AQLs have been met. If the contractor has not met the minimum requirements, it may be asked to develop a corrective action plan to show how and by what date it intends to bring performance up to the required levels.

6.2 Reporting

6.2.1 At the end of each month, the TM will prepare a written report for the COR summarizing the overall results of the quality assurance surveillance of the contractor’s performance. This written report, which consists of the contractor’s submitted monthly report and the completed quality assurance monitoring form (Attachment 2), will become part of the QA documentation. It will enable the government to demonstrate whether the contractor is meeting the stated objectives and/or performance standards, including cost/technical/scheduling objectives.

6.3 Reviews and Resolution

6.3.1 The COR may require the contractor’s project manager, or a designated alternate, to meet with the TM and other government personnel as deemed necessary to discuss performance evaluation. The TM will define a frequency of in-depth reviews with the contractor, including appropriate self-assessments by the contractor. The agenda of the reviews may include:

Monthly performance assessment data and trend analysis

Issues and concerns of both parties Projected outlook for upcoming months and progress against expected trends, including a corrective action plan analysis Recommendations for improved efficiency and/or effectiveness

6.3.2 The TM and contractor should jointly formulate tactical and long-term courses of action. Decisions regarding changes to metrics, thresholds, or procedures should be clearly documented.

ATTACHMENT 1: PERFORMANCE REQUIREMENTS SUMMARY

Performance Standard

Exceptional Very Good Satisfactory Marginal Unsatisfactory

Performance Category: Quality of Product or Service Quality Draft and Final deliverables are of excellent quality, approved as submitted, or with no substantive comments (limited to grammar, spelling, or terminology).

Quality of work performed exceeds the minimum PWS requirement and/or client expectations.

No deficiencies noted.

Draft deliverables are of high quality and comments are mostly minor. Final deliverables are approved after one (1) round of Government comments.

Work performed meets all PWS requirements with no identified deficiencies.

Draft deliverables are of acceptable quality with only a few comments identifying major weaknesses.

Final deliverables are approved after two (2) rounds of Government comments.

meets all PWS requirements with only minor identified deficiencies that do not compromise the quality of the data collected or work performed, and can be corrected.

Draft deliverables are of poor quality with a significant number of comments identifying major weaknesses or deficiencies.

Final deliverables require more than two (2) rounds of Government comments before being approved.

meets minimum

PWS

requirement with identified deficiencies that compromise the quality of the data collected or work performed, but were corrected.

Draft deliverables are of very poor quality and are rejected for resubmittal without comment. Final deliverables did not comply with contract requirements, or one or more document versions required more than three

(3) rounds of Government comments before being approved.

Work performed does not meet

PWS

requirements and/or identified deficiencies compromise the quality of the data collected or work performed, and cannot be corrected.

Performance Category: Schedule

Schedule All deliverables submitted on time, no extensions required for any deliverable submittals.

All deliverables submitted on time, with no more than one deliverable due date extension request.

All deliverables submitted on time, with more than one deliverable due date extension requests.

One deliverable submitted late.

Several deliverables submitted late.

Standard

Exceptional Very Good Satisfactory Marginal Unsatisfactory

Contractor achieves milestones per baseline, maintaining a “green” rating for schedule performance within the

PSSR. *

Contractor achieves milestones per baseline, maintaining mostly a “green” rating for schedule performance within the PSSR, with minimal “yellow” ratings which are quickly resolved.

Contractor achieves milestones, maintaining mostly a “green” rating for schedule performance within the PSSR, with minor “yellow” and/or “red” ratings which are resolved.

Contractor maintains mostly “yellow” rating for schedule performance within the PSSR, with minor “red” ratings which are resolved.

Contractor maintains mostly a “red” rating for schedule performance within the PSSR, with contractor-related issues that are not quickly resolved.

Performance Category: Management of Key Personnel and Resources

Management of Key Personnel and Resources

Zero (0) instances of resource management issues creating a negative impact to the activity.

No more than one (1) instance of resource management issues creating a negative impact to the activity, which was ultimately addressed by the contractor.

No more than two (2) instances of resource management issues creating a negative impact to the activity, which were ultimately addressed by the contractor.

No more than three (3) instances of resource management issues creating a negative impact to the activity or more than one (1) instance of resource management issues that was not satisfactorily addressed by the contractor.

More than three

(3) instances of resource management issues creating a negative impact to the activity or multiple unresolved resource management issues.

Performance Category: Cost Control

Cost Control Contractor achieves milestones per baseline, maintaining a “green” rating for cost performance within the

PSSR.

All PWS requirements

Contractor achieves milestones per baseline, maintaining mostly a “green” rating for cost performance within the PSSR, with minimal “yellow” ratings which are quickly resolved.

Contractor achieves milestones, maintaining mostly a “green” rating for cost performance within the PSSR, with minor “yellow” and/or “red” ratings which are resolved.

maintains mostly a “yellow” rating for cost performance within the PSSR, with minor “red” ratings which are resolved.

maintains mostly a “red” rating for cost performance within the PSSR, with contractor-related issues that are not quickly resolved.

Required significant

Standard

Exceptional Very Good Satisfactory Marginal Unsatisfactory met and exceptional value achieved while finishing under total budget.

All PWS requirements met and exceptional value achieved while finishing within total budget.

All PWS requirements met and finished within total budget.

Required minimal additional funding or scope reduction to complete all requirements.

additional funding to complete all requirements.

Performance Category: Business Relations

Business Relations

Interaction/ communication between the contractor and the government is exceptional; no problems identified.

Subcontracts awarded in a timely manner with exceptional management of subcontractors.

Interaction/ communication between the contractor and the government is very good; no problems identified.

timely manner with very good management of subcontractors.

Interaction/ communication between the contractor and the government is satisfactory;

minor problems identified and adequate corrective actions taken.

reasonable manner with satisfactory management of subcontractors.

Interaction/ communication between the contractor and the government is marginal;

major problems identified and adequate corrective actions taken.

awarded in an untimely manner and/or issues stemming from marginal management of subcontractors.

Interaction/ communication between the contractor and the government is unsatisfactory;

major problems identified and inadequate corrective actions taken.

Multiple issues stemming from poor management of subcontractors.

Performance Category: Utilization of Small Business

Small Business

Usage of small business shall be documented; if no usage then note as NA

Performance Category: Other Areas

Other NA NA NA NA NA

The following guidelines are provided for issuing ratings that are subjective in nature, these ratings will be supported by the weight of evidence documented during the government's surveillance efforts:

Exceptional: Performance meets contractual requirements and exceeds many to the Government's benefit. The contractual performance of the element or sub-element being assessed was accomplished with few minor problems for which corrective actions taken by the Contractor were highly effective.

Very Good: Performance meets contractual requirements and exceeds some to the Government's benefit. The contractual performance of the element or sub-element being assessed was accomplished with some minor problems for which corrective actions taken by the Contractor were effective.

Satisfactory: Performance meets contractual requirements. The contractual performance of the element or sub-element contains some minor problems for which corrective actions taken by the Contractor appear or were satisfactory.

Marginal: Performance does not meet all contractual requirements. The contractual performance of the element or sub-element being assessed reflects a serious problem for which the Contractor has not yet identified corrective actions.

The Contractor's proposed actions appear only marginally effective or were not fully implemented.

Unsatisfactory: Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub-element contains serious problems for which the Contractor's corrective actions appear or were ineffective.

* PSSR is defined as “Program/Project Summary Status Report” and is an earned value based report generated monthly by the Contractor which outlines the cost and schedule performance for each milestone within a task order. The Contractor is required to include each task order’s monthly PSSR within the respective Monthly Status Report. The color system is noted within the PSSR, and is a Green/Yellow/Red System that identifies how progress conforms to the original schedule or cost-tracking module for a specific component, whereby reflecting when variances occur from the Original to the Actual, as follows:

o Green - Vary 10% or less for both Cost and Schedule performance.

o Yellow - Vary between 11 and 19% for either Cost or Schedule performance.

o Red - Vary 21% or more for either Cost or Schedule performance.

ATTACHMENT 2: SAMPLE QUALITY ASSURANCE

MONITORING FORM

TASK NAME/NUMBER: ATC Safety Support Services

SURVEY PERIOD:

ANALYSIS OF RESULTS:

Reference Surveillance Activity Checklist for Assessed Activities versus Performance Standards.

Contractor’s Performance (Check):

Performance Standard

Ex ce pt io na l

Ve ry

G oo d

Sa tis fa ct or y

M ar gi na l

U ns at is fa ct or y

N ot

E va lu at ed

Quality of Product or Service Schedule Management of Key Personnel and Resources

Cost Control Business Relations Other Areas Contractor’s Overall Performance for Period

Narrative of Performance Assessed During Survey Period:

PREPARED BY: ___________________________________ DATE: _________________

ATTACHMENT 3: SURVEILLANCE ACTIVITY CHECKLIST

ATC Safety Support Services

Deliverable Applicable Performance Standard

Quality Schedule Management of Key

Personnel

Cost Control

Business Relations

Utilization of Small Business

Other Areas

Component 1 (PWS Section C3.): Program Management

• PMP/WBS

• Monthly Reports

• Record of

Meetings/Minutes

• Kickoff Meeting

X X X X X X NA

Component 2 (PWS Section C4):

Deliverables.

• A001 Work

Products (audit results / incident reports / presentation briefs)

X X NA

Component 3 (PWS Section C4):

Deliverables.

• B001 Contractor’s

Progress, Status and Management Report.

X X X X X

Component 3 (PWS Section C4):

Deliverables.

• C001 Mandatory

Training Certificates.

X X