2.01 Attachment 3 - Quality Assurance Surveillance Plan 1-11-24.docx

Quality Assurance Surveillance Plan (QASP) 1 Introduction This Attachment provides a summary of the QASP performance requirements for the Department of the Navy – Wireless Services Spiral 4, Indefinite Delivery, Indefinite Quantity Multiple Award Contracts (IDIQ, MACs). The purpose of any QASP is to provide a structured process for managing the Contracts’ performance and reducing performance risk. It emphasizes communication during all phases of performance.

The DON Wireless Services Contracts Contracting Officer’s Representative (COR) will use a matrixed approach to conduct Contract surveillance, including observations from the Task Order COR, Customer, or other Government personnel. Additionally, some aspects of performance will require contributions from the Ordering Office relative to the Task Order administration or similar matters.

1.1 Purpose

This QASP describes the systematic methods used to measure the Contractor’s performance. The QASP additionally provides the Government a means for evaluating whether the Contractor is complying with the terms and conditions of the Contract. The QASP is the primary tool for documenting Contractor performance. The QASP defines roles and responsibilities, identifies the performance objectives, and defines methodologies used to monitor and evaluate Contractor performance, and describes the analysis of quality assurance monitoring results.

Authority for issuance of this QASP is provided at FAR 52.246-4 Inspection of Services-Fixed Price, which provides for inspections and acceptance of the services, and documentation called for in the Contract to be accomplished by the Procuring Contracting Officer (PCO) or their duly authorized representative, the COR. The Government shall perform inspections and tests in a manner that will not delay Contractor performance.

1.2 Quality Performance Surveillance Strategy

The Contractor may develop and utilize their own staffing and procedures, for use on a continuous basis, for self-inspecting each performance objective and other performance requirements outlined in the terms and conditions. It is the Contractors responsibility to internally monitor business processes and practices to ensure performance is managed effectively.

The Contracting Officer and the COR will monitor performance, review performance and any other documentation provided, to determine how the Contractor is performing against the performance objectives. To monitor performance, the Ordering COR and COR will utilize the QASP performance standards and methods of surveillance as indicated within each performance objective table. The Evaluation Rating Definitions (Table 1) shall be utilized to assign appropriate ratings for each performance objective rated.

1.3 Roles and Responsibilities

1.3.1 Contract Ordering Officer

The Ordering Officer ensures performance of all necessary actions for effective contracting, ensures compliance with the Contract terms and conditions, and safeguards the interests of the United States in the contractual relationship. The Ordering Officer is ultimately responsible for determining the adequacy of Department of the Navy Wireless Services Contracts Spiral 4 Attachment 3: Quality Assurance Surveillance Plan (QASP) the Contractor’s performance. The Ordering Officer will consider all information received from Order COR when determining the adequacy of the Contractor’s performance.

The Ordering Officers functions include:

· Monitors, assesses, and communicates the Contractor performance.

· Reviews the Order COR’s performance reports (CPARS report if over threshold or Quality Assurance (QA) report) and approves them.

· Assures that the Contractor receives impartial, fair, and equitable treatment under the Contract.

1.3.2 Task Order Contracting Officer’s Representative (COR)

The Order COR, assigned by the Ordering Officer, performs quality assurance functions for the Contract wireless mobility services. The Order COR uses the QASP to conduct the oversight and surveillance process. The purpose is to ensure that the Contractor meets the performance standards contained in the terms and conditions of the Contract. The Order COR is not empowered to make any contractual commitments or to authorize any contractual change on the Government’s behalf.

The Order COR functions include:

· Provides continuous technical oversight of the Contractor’s performance.

· Assists in acquisition planning.

· Provides technical assistance to the Ordering Officer.

· Maintains proficiency in Order surveillance procedures.

· Performs surveillance in accordance with the QASP and provides Ordering Officer documentation of surveillance observations.

· Notifies the Ordering Officer of any significant performance deficiencies.

· Monitors, assesses and communicates Contractor performance to the Ordering Officer.

· Collect all relevant documentation, to include relevant e-mails and documented phone conversations, and/or Contractor invoices submitted.

· Develops a QA report for each Order and maintains a Quality Assurance file that accurately documents the Contractor’s actual performance.

· For urgent concerns discovered during the Order performance period, or during the QA Report process, where timeliness of Contractor performance concerns need to be processed quickly, the Order COR shall notify the Ordering Officer of any significant performance deficiencies as soon as possible and not wait for all documentation to be collected.

2 Performance Objectives The Contractor shall provide wireless mobility services pursuant to the requirements outlined in the terms and conditions. While the Contractor is fully expected to comply with all requirements, the Government’s assessment of performance will primarily focus on the performance objectives detailed below. The level of acceptance for each objective is established IAW evaluation criteria listed herein for Technical/Quality, Schedule, Management, and Customer Relations, and numeric grades are to be assigned for each ranging from one (1) to five (5). As described in Section 3.1, an overall numeric goal of three (3) is targeted for each performance objective. Ratings of “4 or 5” may be assigned if the Contractor performed single or multiple significant events that were of benefit to the Government, and there were NO significant weaknesses identified. A fundamental principle of assigning ratings is that Contractors will not be assessed a rating lower than three (3) solely for not performing beyond the requirements of the Order.

The Contractors performance objectives are summarized in the following table. However, additional performance element requirements may be included for specific Orders by the Ordering Office at the Order Level.

2.1 Performance Objective #1 – Technical/ Quality

Quality of the wireless mobility services and devices provided shall conform at least to the quality standards contained herein.

Specific performance standards and surveillance methods for quality include, but are not limited to:

2.1.1 Coverage

Performance Element

Performance Standard

Surveillance

Acceptable Quality Level

Voice & Texts Coverage

Within the Geographical Area(s) specified in the subject order in accordance with PWS Sections 6.6.2b and 6.6.3, and for the Radio Access Network(s) specified in the subject order in accordance with PWS Section 6.3, 100% of the devices have the minimum coverage specified for the subject order for either Legacy Coverage as defined and measured in PWS Section 6.6.1, or for Other Level of Service Coverage as defined and measured in PWS Section 6.6.2.

Email Coverage

Within the Geographical Area(s) specified in the subject order in accordance with PWS Sections 6.6.2b and 6.6.3, and for the Radio Access Network(s) specified in the subject order in accordance with PWS Section 6.3, 100% of the devices have the minimum coverage specified for the subject order for either Legacy Coverage as defined and measured in PWS Section 6.6.1, or for Other Level of Service Coverage as defined and measured in PWS Section 6.6.2.

Paging Coverage

2.2 Performance Objective #2 – Schedule

The Schedule Performance Objective assess the timeliness of the Contractor for the Orders, milestones, delivery schedules and administrative requirements.

Specific standards and surveillance methods for schedule include, but are not limited to:

2.2.1 Delivery of Devices

Performance Element

Performance Standard

Delivery of Devices

2.3 Performance Objective #3 – Management

Management assess the integration and coordination of all activity needed to execute the Order, specifically the timeliness, completeness and quality of problem identification, corrective action plans, quotes submittals, the Contractor’s history of reasonable and cooperative behavior (to include timely identification of issues in controversy), customer satisfaction, and timely award.

Specific standards and surveillance methods for management include, but are not limited to:

2.3.1 Invoices

Performance Element

Performance Standard

Invoices Content of invoices are in compliance w/ Contract terms and conditions.

Invoice submissions are timely, accurate, and complete – submit invoices into Wide Area Work Flow (WAWF) in a timely manner (90 days).

Invoice Notifications

2.3.2 Order Modifications

Performance Performance Standard Surveillance Acceptable Quality Level Element

Modifications

the Ordering Officer.

90% of the time email responses/notification are made to the Ordering Officer within one (1) business day.

2.3.3 Reporting

Performance Element

Performance Standard Surveillance

Reporting

DON Wireless Services Contract COR and Order COR.

95% of reports are submitted within the timeframes identified within the Contract requirement terms and conditions..

2.3.4 DON Wireless Services Contract, Spiral 4, Dedicated Website

Performance Performance Standard Surveillance Element

Group – Enterprise Level

Website – General and Order Level

2.4 Performance Objective #4 – Customer Relations

Customer Relations assess the Contractor’s history of reasonable and cooperative behavior (to include timely identification of issues in controversy), customer satisfaction, and timely award.

Specific standards and surveillance methods for customer relations include, but is not limited to:

2.4.1 Customer Service

Performance Element

Performance Standard

Customer Service

Issue Resolution Work with the PCO, DON Wireless Services Contract COR, Ordering Officer, Order COR, end user, and any other Government representative to resolve issues

3 Quality Assurance Monitoring

3.1 Quality Assurance Reporting: To be submitted into SPM (Surveillance Procurement Module) located in PIEE (Procurement Integrated Enterprise Environment)

Contractor performance will be assessed both individually (per Task Order) and annually (a roll-up of quarterly performance). The Order COR will complete a Quality Assurance (QA) report for each Order via SPM, the report shall also be included in each Order, and submitted via email to the DON Wireless Services Contract COR quarterly*. Additionally, the DON Wireless Services Contract COR shall submit an annual QA report to the PCO 60 days before the end of each ordering period to capture a roll-up from the quarterly performance reports within each year.

QASP reporting is completed using surveillance information observed by the Order COR and any other Government notes communicated during the performance period of each Order. The QASP score for a particular Order will be computed as the average of each applicable performance objective listed in Section 2. Similarly, the overall QASP score for the quarter will be computed as the average of all Orders during that quarter. A numeric goal of three (3) (Satisfactory) is expected for each performance objective. An average QASP score for an Order of less than three (3) is considered less than satisfactory performance for that Order. Likewise, an overall average QASP score for the quarter of less than three (3) is considered less than satisfactory performance for the quarter.

*A quarter is defined as a 3-month period of time beginning on the first month of the fiscal year. If the period of performance begins after a quarter begins, the first quarter will include the portion leading up to the first quarter plus the first quarter. The final quarter will be defined as any period of performance left after the last completed quarter.

Annually, or as required, the Government will input performance evaluations into the Department of Defense (DoD) Contractor Performance Assessment Reporting System (CPARS). The Order COR is responsible for inputting performance ratings into CPARS for every Order that exceeds $1M and providing the finalized assessment to the DON Wireless Services Contract COR. In these instances, Contractors are required to obtain CPARS accounts for electronic distribution of assessments. CPARS accounts can be established at https://cpars.gov.

3.2 Quality Assurance Results

The PCO will provide the QA Annual Report to the Contractor, via email, within 30 days after the end of each ordering period. Briefings regarding Contractor performance, via teleconference meeting, will be discussed during CONTRACT Annual reviews and/or Progress Reviews.

The PCO/Ordering Officer may require the Contractor’s Dedicated Project Manager, or a designated alternate, to meet with the DON Wireless Services Contract COR as deemed necessary to discuss the performance evaluation. The Order COR/DON Wireless Services Contract COR will conduct in-depth reviews with the Contractor as needed.

However, if the need arises, the Contractor will meet with the PCO/Ordering Officer, as often as required, to formulate tactical and long–term courses of action. The agenda of the reviews may discuss:

· Performance measured by the metrics and trends

· Issues and concerns of both parties

· Recommendations made by the Government based on feedback and assessments

· Issues arising from independent reviews and inspections

3.3 Negative Performance

The PCO will evaluate the QA Annual Report results should performance not meet the QASP standards. As determined by the PCO, in the event the Contractor fails to perform, the Contractor may be barred from being awarded additional Orders for a specified period and/or may receive negative past performance ratings.

If the Contractor's overall performance is unsatisfactory, as documented within the quarterly evaluations, the PCO and COR will provide in writing a report identifying the issues that have resulted in the performance being evaluated as “unsatisfactory”. In response, the Contractor shall provide the below information to the PCO and COR, in writing within five (5) calendar days of receipt of the report:

· Reason(s) for unsatisfactory performance,

· Action(s) the Contractor proposes to take to remedy the deficiencies; and,

· Any measures the Contractor plans to adopt to ensure that similar deficiencies will not reoccur in the future.

Based upon the Contractor’s past performance and their response to the aforementioned information, the PCO will determine if any further action(s) need to be taken.

3.3.1 Off Ramping

Contractor is rated less than “satisfactory” for multiple performance objectives during QA reports or CPARS assessments throughout a quarterly reporting period, the following may apply:

a. Off-ramped from the DON Wireless Services Contracts

· During the Contract annual review, if the PCO determines that the Contractor has failed to perform satisfactorily, the Contractor may be off-ramped.

Table (1)

Evaluation Rating Definitions

Exceptional Performance meets Contract terms and condition requirements and exceeds many to the Government’s benefit. The contractual performance of the element or being evaluated was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

To justify an Exceptional rating, identify multiple significant events and state how they were of benefit to the Government. A singular benefit, however, could be of such magnitude that it alone constitutes an Exceptional rating. Also, there should have been NO severe issues identified.

Very Good

Performance meets Contract terms and condition requirements and exceeds some to the Government’s benefit. The contractual performance of the element being evaluated was accomplished with some minor problems for which corrective actions taken by the Contractor was effective.

To justify a Very Good rating, identify a significant event and state how it was a benefit to the Government. There should have been no severe issues identified.

Satisfactory

Performance meets Contract terms and condition requirements. The performance of the element contains some minor problems for which corrective actions taken by the Contractor were appropriate or were satisfactory.

To justify a Satisfactory rating, there should have been only minor problems, or major problems the contractor recovered from without impact to the Contract/Order. A fundamental principle of assigning ratings is that Contractors will not be evaluated with a rating lower than Satisfactory solely for not performing beyond the requirements of the Contract/Order. Vendors that meet acceptable performance will receive a rating of satisfactory.

Marginal

Performance does not meet some Contract term and condition requirements. The performance of the element being evaluated reflects a serious problem for which the Contractor has not yet identified corrective actions. The Contractor’s proposed actions appear only marginally effective or were not fully implemented.

To justify Marginal performance, identify a significant event for each objective that the Contractor had trouble overcoming and state how it impacted the Government

Unsatisfactory

Performance does not meet most Contract term and condition requirements and recovery is not likely in a timely manner. The performance of the element contains a serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

To justify an Unsatisfactory rating, identify multiple significant events in each category that the Contractor had trouble overcoming and state how it impacted the Government. A singular problem, however, could be of such serious magnitude that it alone constitutes an unsatisfactory rating.