Attachment 0003 Quality Assurance Surveillance Plan.pdf

QASP JTT- IBS

Solicitation #: SPRBL1-21-R-0007

16 October 2019

QUALITY ASSURANCE SURVEILLANCE PLAN

For the

JOINT TACTICAL TERMINAL Integrated Broadcast Service (JTT-IBS)

Task Order #: XX

Prepared By David Farmer for the

U.S. Army Communications Electronics Command (CECOM) Integrated Logistics Support Center (ILSC)

Intelligence Electronics Warfare & Sensors (IEW&S)

1.0 Introduction

This Quality Assurance Surveillance Plan (QASP) sets forth the procedures and guidance that the Contracting Officer’s Representative (COR) will use in evaluating the technical and quality performance of the contractor in accordance with the terms and conditions of the Performance Work Statement (PWS). A copy of the signed final plan will be furnished to the contractor so that the contractor will be aware of the methods that the COR will use in evaluating performance of this PWS.

2.0 Purpose

The purpose of the QASP is to assure that the performance of specific activities and the completion of projected milestones are accomplished in accordance with all requirements set forth in the PWS. This QASP describes the mechanism for documenting noteworthy accomplishments or discrepancies for work performed by the contractor. Information generated form the COR’s surveillance activities will directly feed into performance discussions with the contractor. The intent is to ensure that the contractor performs in accordance with performance metrics set forth in the contract documents, the Army receives the quality of service called for in the contract, and the Army only pays for acceptable services received.

The QASP is intended to accomplish the following:

• Define the roles and responsibilities of participating Army officials.

• Define the key milestones, deliverables, and standards that will be assessed.

• Describe the surveillance methodology that will be employed by the

Army in assessing the contractor’s performance.

• Describe the surveillance documentation process and provide copies of the forms that the Army will use in evaluating the contractor’s performance.

• Define: Exceptional, Very Good, Satisfactory, Marginal and Unsatisfactory performance standards for key milestones, deliverables, and standards.

• Outline corrective action procedures.

3.0 Performance Management

This Quality Assurance Surveillance Plan (QASP) will define the performance management approach to monitor, manage and take appropriate action on the contractor’s performance against expected outcomes or performance objectives communicated in the Performance Work Statement (PWS). Performance management rests upon developing a capability to review and analyze information generated through performance metrics. The ability to makes decision based on the analysis of performance data in the cornerstone of performance management. The data generated in a performance management approach provides information that indicates whether or not expected outcomes for required services are being achieved adequately by the contractor.

Performance management focuses on assessing whether or not outcomes are being achieved and migrates away from scrutiny on compliance with the processes and practices used to achieve the outcome. The only exceptions to process reviews are those required by law (International, Federal, State and local) and compelling business situations such as safety and health. An outcome focus provides the contractor flexibility to continuously improve and innovate over the course of the contract as long as the critical outcomes expected are being achieved at the desired levels of performance.

4.0 Performance Management Strategy

The contractor’s internal quality control system will set forth the staffing and procedures for self-inspecting the quantity, quality, timeliness, responsiveness, customer satisfaction and other performance requirements as stated in the PWS.

The contractor will use its internal quality control system to assess and report their performance to the designated Government representative, the COR.

The Government representative will monitor performance and review performance reports furnished by the contractor to determine how the contractor is performing against communicated performance objectives. The Government will make decisions based on performance-based service metric data and notify the contractor of those decisions. The contractor will be responsible for making required changes in processes and practices to ensure performance is managed effectively.

This QASP is a “living document”, and the Government may review and revise it on a regular basis. However, the Government shall coordinate changes with the contractor. Copies of the original QASP and revisions shall be provided to the contractor and Government officials’ implementing surveillance activities.

5.0 Scope

The purpose of this QASP is to describe the methodology for Government personnel to evaluate and assess contractor performance under this contract.

The Government will assess the contractor's performance related to the delivery of services under the contract. The contractor's performance for the delivery of services will be reviewed based on each task order issued. The metric for task orders will vary based on the task to be accomplished.

Overall performance on the basic contract will be reviewed quarterly between the Contracting Officer Representative (COR), Contracting Officer (KO)[as needed], and the contractor’s personnel, which may include the Program Manager, Cost Manager and Technical Experts. This is a FFP, CPFF, Indefinite Delivery, Indefinite Quantity (IDIQ) services contract.

6.0 Roles and Responsibilities of Quality Assurance Army Officials

6.1 Contracting Officer (KO)

The Contracting Officer (KO) has overall responsibility for overseeing the contractor's performance. The KO is responsible for the day-to-day monitoring of the contractor's performance in the areas of contract compliance, and contract administration; reviewing the COR's assessment of the contractor's performance;

and resolving all differences between the COR's assessment and the contractor’s assessment of performance. It is the KO that assures the contractor receives impartial, fair, and equitable treatment under the contract. The KO is ultimately responsible for the final determination of the adequacy of the contractor's performance.

6.2 Contracting Officer Representative (COR)

The Contracting Officer's Representative (COR) is responsible for technical administration of the project and assures proper Army surveillance of the contractor's performance. The COR is responsible for monitoring, assessing, recording, and reporting on the technical performance of the contractor on a day-to-day basis.

6.3 Technical Experts and Subject Matter Experts (SME)

The KO and COR are responsible for assessing the contractor’s performance, with input from other Government personnel. Government personnel assessing the contractor's performance may include the Team Leader, Product Manager, Product Support Integrator (PSI), Defense Contract Management Agency (DCMA), technical experts and other SMEs aid in the support of the system. The COR is responsible for completing monthly performance evaluations. The frequency of these evaluations may change as the quality of the contractor’s performance changes.

The address for the Government responsible for this program is given below:

U.S. Army Communications Electronics Command Integrated Logistics Support Center Directorate for Intelligence, Electronics Warfare and Sensors

ATTN: AMSEL-LCW-SD

Aberdeen Proving Ground, Maryland 21005

7.0 Method of Assessment

For all Task/Delivery Orders, work performance will be assessed through the following areas of evaluation: Technical, Cost and Schedule.

The Program Office will evaluate the contractor's performance against the schedule and the variance in actual cost from estimated costs. Integral with the evaluation is an examination of the contractor's ability to meet the delivery/performance schedule and execute within the proposed/estimated cost.

To determine the ratings, inputs from multiple Government personnel may be used.

8.0 Procedures

The COR will review the data submitted during a quarterly meeting with the contractor. The monthly Repair Status Report from the contractor will serve as the trigger for this meeting. Meeting will be conducted via teleconference with contractor and Government representatives or on site at either Government or contractor facilities. The KO will participate in these meetings as needed.

8.1 Technical

Technical performance will be reviewed quarterly through meetings between the COR, other Government personnel, and the contractor’s Program Manager.

Factors to be monitored are technical support and product quality according to PWS paragraphs 3.11 and 3.5.1, respectively. Any deficiencies/issues will be reported to the KO and documented.

8.2 Schedule

The contractors' schedule performance with regard to deliveries will be reviewed and evaluated by the Government personnel and the COR. Any deficiencies/issues will be reported to the KO in documented meeting minutes.

The contractor will be rated based on repair turnaround times (RTAT). The Contractor shall be required to meet an RTAT of 60 days not including administrative time unless this requirement is waived by the Government according to section 3.4.3 of the PWS.

8.3 Cost

The primary factor to be considered is successful completion of task orders within the originally awarded cost. The COR will provide the contractor Program Manager and the KO with any concerns/comments.

8.4 Documents/Reporting

The contractor’s Monthly Repair Status Reports are to be prepared and submitted according to PWS paragraph 3.6 in accordance with CDRL A001.

These reports will be reviewed, in addition to other contractor documents, technical support feedback, information, and observations/experiences of the contractor by Government personnel. Government program personnel have open access to the KO and COR, and may provide both negative and positive feedback on contractor performance at all times.

The COR will be responsible for reporting the ratings and compliance results to the KO. As a basis for the report, the COR will use the contractor's delivery performance for the contractor's execution of services, as procured via task orders, to include effectiveness in performing the required tasks, performing the tasks within the period of performance specified in the task order, and completing the required tasks within the negotiated cost of the task order. The Government will not hold the contractor responsible for failures in satisfying the technical performance, schedule performance, and cost performance that are due to actions or inactions by the Government. Any issues which cannot be resolved at the working level or which impact on contractual requirements shall be reported to the KO by the COR.

9.0 Performance Standards.

The Contractor's performance will be evaluated by assessing the key milestones and deliverables above according to the standards of Quality, Schedule, Management of Key Personnel and Resources, and Stakeholder Concurrence, and Security in accordance with Table 1, below. For each of these performance standards, the COR will assign one of five ratings of the Contractor's performance: exceptional, very good, satisfactory, marginal, or unsatisfactory, as shown in Table 1. Performance Rating definitions are provided below.

Exceptional: Performance meets contractual requirements and exceeds many to the Government's benefit. The contractual performance of the element or sub-element being assessed was accomplished with few minor problems for which corrective actions taken by the Contractor were highly effective.

Very Good: Performance meets contractual requirements and exceeds some to the Government's benefit. The contractual performance of the element or sub-element being assessed was accomplished with some minor problems for which corrective actions taken by the Contractor were effective.

Satisfactory: Performance meets contractual requirements. The contractual performance of the element or sub-element contains some minor problems for which corrective actions taken by the Contractor appear or were satisfactory.

Marginal: Performance does not meet all contractual requirements. The contractual performance of the element or sub-element being assessed reflects a serious problem for which the Contractor has not yet identified corrective actions. The Contractor's proposed actions appear only marginally effective or were not fully implemented.

Unsatisfactory: Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub-element contains serious problems for which the Contractor's corrective actions appear or were ineffective.

10.0 Rating Areas and Metrics.

The required performance standards and/or quality levels are included in the PWS. The Government will evaluate and rate the Contractor's performance in the areas cited in the table below. To determine the ratings, inputs from multiple sources may be used (e.g., PRD Engineer, ILSC Item Manager, ILSC Provisioner, PM DCGS-A and Operators and Maintainers of the system, etc.).

Although the Contractor may provide comments regarding the inability to meet the standards specified for a given rating, the Government unilaterally determines the rating for the Contractor.

Area PWS Para.

Criteria Exceptio nal

Very Good

Satisfact ory

Margina l

Unsatisfact ory

Scientific and Technical Reports

3.11.

Monthly 0-10 calendar days after end of month

10-20 calendar days after end of month

20-30 calendar days after end of month

30-40 calendar days after end of month

41 calendar days or more after end of month

Governme nt Furnished Equipmen t Repair Status Report

3.6 submitted every 30 days calendar days or less

30 to 35 calendar days

35 to 40 calendar days

40 to 45 calendar days

46 calendar days or more

Test and Inspect

3.3.2 30 days 15 calendar days or less calendar days or less calendar days

31 to 45 calendar days

46 calendar days or more

Test and Inspect on complex repair

3.3.2 60 days 30 calendar days or less calendar days or less calendar days

61 to 75 calendar says

76 calendar days or more

Repair Turn- Around Time

(RTAT)

3.4.3 TBD 120

calendar days or less

120-130 calendar days or less

130-140 calendar days or less

140-150 calendar days or less

151 or greater calendar days

Performan ce and Cost Report

3.10.

Monthly 0-10 calendar days after end of month

10-20 calendar days after end of month

20-30 calendar days after end of month

30-40 calendar days after end of month

41 calendar days or more after end of month

Table 1 – Rating Areas and Metrics

11.0 Acceptance Criteria.

The Contractor shall specify the acceptance criterion/criteria to be used for production and repair items to be delivered under this contract. The acceptance criterion/criteria for each item shall include at least one of the following:

11.1 Certificate of Conformance - according to FAR 52.246-15.

11.2 Analysis - proving the acceptability of an item via a report; Contractor shall obtain prior approval from the Contracting Officer for this approach.

11.3 Visual Inspection - confirming no observable improprieties; Contractor shall obtain prior approval from the Contracting Officer for this approach.

11.4 Demonstration - confirming form, fit, and function without measurements; Contractor shall obtain prior approval from the Contracting Officer for this approach.

11.5 Acceptance Test – Contractor shall utilize DD Form 250 to verifying form, fit, and function with measurements; Contractor shall follow the verification policy established by the Defense Contract Management Agency (DCMA) for this approach.

Note: For each “make” item, the contractor shall utilize an Acceptance Test.

12.0 Methodologies to Monitor Performance

12.1 Surveillance Techniques

In an effort to minimize the performance management burden, simplified surveillance methods shall be used by the government to evaluate contractor performance when appropriate. The primary methods of surveillance are Periodic Inspections and customer feedback. The COR will perform the periodic inspection on a monthly basis. Further the COR will rate and complete monthly quality assurance surveillance plan checklists on the contractor’s performance in the Wide Area Work Flow (WAWF), Contractor Officer Representative Tool (CORT) module.

12.2 Customer Feedback

The contractor is expected to establish and maintain professional communication between its employees and customers. The primary objective of this communication is customer satisfaction. Customer satisfaction is the most significant external indicator of the success and effectiveness of all services provided and can be measured through customer complaints.

Performance management drives the contractor to be customer focused through initially and internally addressing customer complaints and investigating the issues and/or problems but the customer always has the option to communicate complaints to the COR, as opposed to the contractor.

Customer complaints, to be considered valid, must set forth clearly and in writing the detailed nature of the complaint, must be signed, and must be forwarded to the COR. The COR will accept those customer complaints and forward to the KO for investigation and guidance.

Customer feedback may also be obtained either from the results of formal customer satisfaction surveys or from random customer complaints.

12.3 Acceptable Quality Levels

The acceptable quality levels (AQLs) included in Table 1 - Rating Area and Metrics for contractor performance are structured to allow the contractor to manage how the work is performed while providing negative incentives for performance shortfalls.

13.0 QUALITY ASSURANCE DOCUMENTATION

13.1 The Performance Management Feedback Loop

The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed in the PWS and are assessed using the performance monitoring techniques outlined in paragraph 4.0 of this QASP.

13.2 Monitoring Medium

The government’s QA surveillance, accomplished by the COR, will be reported and filed via monthly quality assurance surveillance checklists in the Wide Area Work Flow Contractor Officer Representative Tool (CORT) module. The checklists, when completed, will document the government’s assessment of the contractor’s performance under the contract to ensure that the required results are being achieved.

14.0 Analysis of Quality Assurance Assessment

14.1 Determining Performance

14.1.1 Government will use the monitoring methods cited to determine whether the performance standards have been met. If the contractor shall not meet the minimum requirements, the KO may request the contractor to develop a corrective action plan to show how and by what date the contractor intends to bring performance up to the required levels.

14.2 Reporting

14.2.1 At the end of each month, the COR will prepare a written report summarizing the overall results of the quality assurance surveillance of the contractor’s performance. This written report, which includes the contractor’s submitted monthly report and the completed quality assurance surveillance checklists, will become part of the QA documentation on the CORT module. This will enable the government to demonstrate whether the contractor is meeting the stated objectives and/or performance standards, including cost/technical/scheduling objectives.

14.3 Reviews and Resolution

14.3.1 The KO may require the contractor’s project manager, or a designated alternate, to meet with the COR and other government IPT personnel as deemed necessary to discuss performance evaluation. The KO will define a frequency of in-depth reviews with the contractor, including appropriate self-assessments by the contractor; however, if the need arises, the contractor will meet with the COR as often as required or per the contractor’s request. The agenda of the reviews may include:

Monthly performance assessment data and trend analysis Issues and concerns of both parties Projected outlook for upcoming months and progress against expected trends, including a corrective action plan analysis Recommendations for improved efficiency and/or effectiveness

14.3.2 The QAR must coordinate and communicate with the contractor to resolve issues and concerns regarding marginal or unacceptable performance.

14.3.3 The Program Manager and contractor should jointly formulate tactical and long-term courses of action. Decisions regarding changes to metrics, thresholds, or service levels should be clearly documented.

Changes to service levels, procedures, and metrics will be incorporated as a contract modification at the convenience of the PCO/ACO.