QASP.pdf

Radio Frequency Interference Monitoring System (RFIMS)

Quality Assurance Surveillance Plan

Version 1.0 [Final]

February 22, 2017

U.S. Department of Commerce (DOC)

National Oceanic and Atmospheric Administration (NOAA)

National Environmental Satellite, Data, and Information Service (NESDIS)

Office of Satellite Ground Services (OSGS)

RFIMS Quality Assurance Surveillance Plan (QASP) RFIMS-CO-PLN-20 ii

QUALITY ASSURANCE SURVEILLANCE PLAN

FOR

The RFIMS

Prepared By: ____________________________

William J. Barnes RFIMS Senior Acquisition Support Project Lead, OSGS

Approved By: ____________________________

Steven Grippando

RFIMS Project Manager, OSGS Systems Engineering Deputy Division Chief iii

CHANGE RECORD

DOCUMENT TITLE: RFIMS Quality Assurance Surveillance Plan (QASP)

VERSION DATE PAGES AFFECTED DESCRIPTION

0.1A 30 November 2016 Initial Draft

1.0 22 February 2017 Final

The document version number identifies whether the document is a working copy, final, revision, or update, defined as follows:

• Working copy or Draft: a document not yet finalized or ready for distribution; sometimes called a draft. Use 0.1A, 0.1B, etc. for unpublished documents.

• Final: the first definitive edition of the document. The final is always identified as Version 1.0.

• Revision: an edition with minor changes from the previous edition, defined as changes affecting less than one-third of the pages in the document. The version numbers for revisions 1.1 through 1.9, 2.1 through 2.9, and so forth. After nine revisions, any other changes to the document are considered an update. A revision in draft, i.e. before being re-baselined, should be numbered as 1.1A, 1.1B, etc.

• Update: an edition with major changes from the previous edition, defined as changes affecting more than one-third of the pages in the document. The version number for an update is always a whole number (Version 2.0, 3.0, 4.0, and so forth).

iv

Table of Contents

1. Introduction

1.1. Purpose

1.2. Scope

1.3. Performance Management Approach

2. Performance Management Criteria

2.1. Methods of Performance Monitoring

2.1.1. Surveillance

2.1.2. Communications and Customer Feedback:

2.1.3. Customer Satisfaction:

2.1.4. Trend Analysis:

2.1.5. Formal Project Reviews:

3. QASP Criteria Performance Metrics:

4. Reporting

5. Reviews and Resolution

Appendix A: Acronym List

Attachment 1: Quality Assurance Monitoring Form

1. Introduction This Quality Assurance Surveillance Plan (QASP) is developed pursuant to the performance requirements contained in the Radio Frequency Interference Monitoring System (RFIMS) Statement of Objectives (SOO).

The QASP sets forth the procedures and guidelines the assigned RFIMS Contracting Officer’s

Representative (COR) and project office staff will use in ensuring the contract performance standards and service levels are achieved through effective and ongoing performance monitoring. The QASP was developed according to Federal Acquisition Regulations (FAR) 37.604 and 46.4 criteria and will be managed by the project office.

1.1. Purpose

The QASP product/service requirements and performance standards (metrics) are applicable to achieving the functional and operational standards, thresholds, and objectives outlined in the SOO. The SOO was developed compliant with Federal Acquisition Regulations (FAR) Part 11.002(2)(i)(A) and (B) and pursuant to FAR 37.602(c) and released with the RFIMS solicitation.

The contractor’s Performance Work Statement (PWS) is prepared in response to the Government SOO per FAR 37.602 requirements. It describes the work effort to be performed under the contract in terms of required “results” or performance “outcomes”, and accordingly, is incorporated as a binding contractual performance requirement.

1.2. Scope

The contractor is responsible for the quality of all work performed. The contractor measures that quality through their own Quality Control Plan (QCP) tailored to the contract PWS, while the Government’s role is ensuring proper Quality Assurance per FAR 46.401. The QCP details the methods, controls, metrics, and measures that will be utilized by the contractor to ensure performance, quality, and timeliness of all products/services to be performed are delivered according to established standards. The QCP is accepted by the Government at time of award and is also incorporated as a binding contractual requirement.

The QASP provides the systematic methods the Government will use to monitor the contractor’s performance and evaluating whether the contractor is meeting the performance standards/quality levels identified in the SOO and incorporated into the PWS as well as ensuring contractor compliance with the

QCP. Further, it identifies the performance objectives, defines the methodologies used to monitor and evaluate the contractor’s performance, and describes quality assurance documentation requirements and the method of analysis (e.g., method of surveillance) to document quality assurance monitoring results.

2. Performance Management Approach This QASP outlines the performance management approach necessary to monitor and manage the contractor’s performance to ensure the expected outcomes or performance objectives communicated in the SOO, and incorporated into the PWS, are achieved. The performance management approach defined herein provides the Government with the capability to review and analyze information generated thorough contractor performance assessment throughout the term of the contract.

Decisions made regarding the acceptability of the contractor’s performance and deliverables are based on factual data that are validated. Based on this analysis, information on performance on the project are documented, communicated, and used for contractor corrective action, as applicable, and can be used as support for other contractual remedies that may be imposed by the Contracting Officer (CO).

The QASP defines the criteria and Acceptable Quality Levels (AQL) the contractor must continuously achieve for acceptable performance and acts as a mechanism for product/service acceptance or rejection criteria. It serves as the basis for formal evaluation and rating of the contractor’s performance for the record. Accordingly, the QASP serves, to a great extent, as the formal basis upon which the Contractor’s

Performance Assessment Reporting System (CPARS) evaluation rating are assigned in accordance with

FAR 42.15. CPARS will include separate subfactors of Quality Control Program per FAR 42.1503(b)(2).

If the contractor has not met the minimum performance requirements, it may be asked to develop a formal Corrective Action Plan (CAP) demonstrating how and by what date it intends to bring performance up to the required levels. CAPs must be submitted to the COR and CO for approval within 5 working days from the time notification is provided to the contractor that a formal CAP is required. The CO will provide written approval or rejection of the CAP within 5 working days after the submission of the CAP.

Resubmission will be required within 3 working days following the date of notification of the rejection is provided by the CO. Failure to meet and AQL and/or implement an effective CAP may result in disincentives being imposed at outlined in Section 2.2, such as fee reductions under Cost Plus Fixed Fee

(CPFF) Contract Line Item Numbers (CLIN) as well as other penalties or contractual remedies being pursued. Conversely, if the contractor exceeds the performance standard, they may be considered for performance incentives also outlined under Section 2.2.

3. Performance Management Criteria

3.1. Methods of Performance Monitoring

3.1.1. Surveillance

The primary methods of surveillance include, but are not limited to:

A. Monitoring deliverable and progress tracking schedules.

B. Random sampling and unscheduled testing may be performed at the contractor’s facility by the COR and/or designated inspector.

C. 100% review and inspection of all data and product deliverables including product physically installed at Government installations.

D. Periodic inspections.

E. Direct observations.

F. Data validation.

G. Additional surveillance methods may be employed as determined necessary based on contractor performance. This could include assignment of an on-site quality assurance inspector/monitor and/or third-party audits.

3.1.2. Communications and Customer Feedback:

The contractor is expected to establish and maintain effective, ongoing, and professional communications between its employees, subcontractors, and Government customers. The contractor is expected to develop and submit a Communications Management Plan (CMP) for Government approval.

The primary objective of ensuring a high level of effective communications is to ensure superior customer awareness at all times and leading to improved satisfaction. The timeliness and effectiveness of the contractor’s communications will be tracked, evaluated, and reported. Communications will take the following forms:

A. Face-to-face meetings – as required

B. Reports – as required

C. Email – daily, as required

D. Briefings – monthly, as required

E. Correspondence – as required

F. Teleconference – minimum of weekly

3.1.3. Customer Satisfaction:

Customer satisfaction is the most significant indicator of success and effectiveness of all services provided by the contractor. Customer feedback will be primarily communicated in the form of email, verbally, and surveys. Formal written customer satisfaction surveys will be generated quarterly during the first 39 months of performance, then annually throughout the remainder of the period of performance.

Customer feedback may take the form of complaints, compliments or other expressions of project concern. Customer satisfaction survey results are important feedback for the record. Contractor’s should use the results to continuously monitor and improve their customer satisfaction standing, ensure contract compliance, and employ effective corrective action strategies. Top objectives include:

A. Correct any meaningful problems that have occurred for customers with the company’s products or service.

B. Use as a mechanism to improve the customer-contractor relationship and communications.

C. Use to improve the contractor’s processes, procedures, and management practices.

D. Gain a greater understanding and insight into the customer’s standards, motivations or drivers, and expectations.

E. Important customer experience feedback to ensure customer expectations are being met.

F. Determine if the contractor’s performance (including management performance) is being viewed favorably.

G. Gauge customer’s perception of the company reputation and potential for repeat business.

Customer satisfaction evaluation results will use the same definitions used for CPARS found at FAR 42.15, Table 42-1 “Evaluation Rating Definitions” and will be reflected under the “Management or Business

Relations” factor in the annual assessment.

3.1.4. Trend Analysis:

Trend analysis will used to assess the contractor’s ongoing cost and schedule performance. Trend analysis techniques to be employed under the contract includes Earned Value Management System (EVMS) data analysis submitted under data deliverables detailed under the SOO.

3.1.5. Formal Project Reviews:

Project reviews are one of the principle mechanisms by which the quality of deliverables, performance of the management process and the on-going viability of the work are assured. Project reviews assist the

Government customer in making ongoing business decisions regarding the project by evaluating its overall health at any given moment in time by:

A. Evaluating the effectiveness of project management.

B. Comparing what was actually delivered against the original requirements.

C. Identifying lessons learned.

D. Assessing performance, e.g. comparing the planned schedule against the actual schedule.

E. Capturing stakeholders’ opinion of how the project was delivered.

F. Disseminating project status results and findings for the record.

The COR is the primary Government representative responsible for overall monitoring and reporting on the contractor’s performance. The Government Project Manager (PM) requires the contractor’s project manager to meet the CO and other project personnel and stakeholders (as deemed necessary) to discuss performance evaluation based upon the results of the COR’s surveillance activities. The PM and the CO will define a frequency (typically monthly) of in-depth project reviews with the contractor that will include the contractor’s own self-assessment of performance. The contractor may request a review cycle that may be more frequent (e.g., but not more frequently than weekly subject to CO approval) if they feel circumstances dictate increased interaction and based on significant status updates. Project reviews may be conducted at the Government’s designated site or at the contractor’s facilities. The agenda of the reviews may include:

A. Monthly performance assessment data and trend analysis.

B. Issues and concerns of both parties.

C. Risk status updates.

D. Projected outlook for upcoming progress against expected trends, including corrective action plan analysis.

E. Recommendations for improved efficiency and/or effectiveness.

F. Issues arising from performance monitoring processes.

4. QASP Criteria Performance Metrics:

The contractor is responsible for the quality of all work performed their personnel and subcontractors and shall guarantee all workmanship and productivity of its personnel. The contractor is responsible for achieving the Acceptable Quality Levels (AQL) for all of the major processes identified below. The

Government will perform continuous quality assurance reviews and monitoring of the contractor’s compliance with the standards through the various surveillance methods identified in this QASP. The

Government will document quality assurance findings on Attachment 1, Quality Assurance Monitoring

Form. Indicators of negative performance, poor quality, and/or deliverable rejection will be documented on the Contractor Deficiency Report (CDR) section of Attachment 1. Incentives and disincentives will be imposed based on the contractor’s performance and adherence to the following criteria:

Product/Service Requirement – SOO Reference

Performance Standard

Acceptable Quality Level

(AQL)

Method of Surveillance

Remediation/Disposition

1. Contractor Quality Control Plan (QCP)

Full (100%) implementati on and

Compliance with approved QCP

Periodic Inspection (minimum 4

[I] When consistently met or exceeded annually, annual CPARS subfactor management of approved

QCP

accepted at time of contract award.

>98% of the time.

times annually).

rating of Very Good or Exceptional.

[D] Standard not met 2 or more times annually, contractor considered contractually non-compliant. CAP required and annual CPARS subfactor rating no higher than Marginal. Other contractual remedies considered by Contracting Officer.

2. Data Deliverables

100% of deliverables are submitted timely and accepted by the Government upon initial submission.

>90% are accepted upon initial submission and are considered timely.

100% acceptable upon first resubmission.

100% review and inspection of all data.

[I] Contract compliance considered favorably for customer satisfaction.

[D] Standard not met 3 or more times annually, Fixed Fee under CPFF reduced by 1%.

3. Cost and Schedule Performance

Strict adherence to negotiated contract cost and performance schedules for each CLIN.

> 95% cost control demonstrated on at the CLIN level.

100% schedule adherence demonstrated at the CLIN level.

Monthly Trend Analysis of EVMS data and other cost and schedule reporting requirements.

[I] Consistently positive SPI (> 1) and CPI (> 1) indicators viewed favorably toward Government consideration of Option exercise. When consistently met or exceeded on each CLIN, annual CPARS Cost Control and/or Schedule/ Timeliness factor rating of Very Good or Exceptional.

[D] Consistent or significant negative SPI (<

1) and CPI (<1) indicators viewed unfavorably toward Government consideration of Option exercise. CAP required and annual CPARS Cost Control and/or Schedule/ Timeliness factor rating no higher than Marginal.

4. Customer Satisfaction

No customer complaints with high levels of satisfaction reported.

Contractor has established positive working relationship with Government customer that is maintained.

Areas of concern/ dissatisfaction are resolved to the Government’s satisfaction within 15 business days and maintained.

Results of ongoing dialog and satisfaction survey results.

[I] Consistently high levels of satisfaction viewed favorably toward Government consideration of Option exercise. When consistently high levels are maintained annually, annual CPARS Management or Business Relations factor rating of Very Good or Exceptional.

[D] Negative satisfaction levels viewed unfavorably toward Government consideration of Option exercise. CAP required.

Annual CPARS Management or Business Relations factor rating could be rated no higher than Marginal if CAP unsuccessful.

5. Program Management Reports and Reviews.

100% accuracy presented in contract performance and risk management status.

>98% accuracy in reporting that demonstrates program outcomes are being properly managed and controlled.

Effectiveness of Program Management Function is validated through survey methods, performance monitoring and communication and feedback.

[I] Standard consistently met is viewed favorably toward Government consideration of Option exercise. When consistently high levels are maintained annually, annual CPARS Management or Business Relations factor rating of Very Good or Exceptional.

[D] Negative satisfaction levels viewed unfavorably toward Government consideration of Option exercise. CAP required.

Annual CPARS Management or Business Relations factor rating could be rated no higher than Marginal if CAP unsuccessful.

6.

Communications.

Timely and effective.

Approved Communications Management

Determined by the Project Manager and

[I] Maintaining good business relations enhances company

Plan (CMP) followed 100% of the time.

Contracting Officer.

reputation and customer satisfaction.

[D] Any variance to AQL, a CAP required. If CAP is ineffective, escalation of matter to Contracting Officer for consideration of contractual remedies.

5. Reporting The COR will make continual reporting to the PM, or at a minimum, at least weekly and formally to the

PM and CO at least monthly. The reporting will summarize the overall results of the quality assurance surveillance activities of the contractor’s performance. This written report, which includes the contractor’s submitted monthly report and the completed quality assurance monitoring forms

(Attachment 1, Quality Assurance Monitoring Form) will summarize and identify deliverables received, milestones achieved, schedule and cost matters, the status of risks, and any outstanding performance concerns along with their apparent root cause and the status of corrective action measures taken to-date.

This becomes the official ongoing record of quality assurance documentation for the project. It enables the Government to determine that the contractor is demonstrating when they are meeting the stated objectives and/or performance standards, including cost/schedule/technical objectives.

6. Reviews and Resolution The COR must coordinate and communicate with the contractor to resolve performance issues or concerns. The PM, COR and contractor should jointly formulate and document long-term courses of action. Decisions regarding changes to metrics, thresholds, or service levels should be clearly documented and must be approved by the CO and incorporated into the contract via a modification.

If the contractor disputes a surveillance determination or has concerns over the Government’s performance management of the contract in general, they must first attempt to resolve through the project office. If, in the contractor’s opinion, resolution is not possible or there is clearly impasse that is adversely affecting the contractual relationship and/or performance, they must report their concerns to the CO within 3 business days from the date they determine resolution through the project office is not possible. The CO will be responsible to determining or negotiating the resolution approach and timeframe.

Appendix A: Acronym List AQL – Acceptable Quality Level

CAP – Corrective Action Plan

CLIN – Contract Line Item Number

CDR – Contractor Deficiency Report

CO – Contracting Officer

COR – Contracting Officer’s Representative

CPARS – Contractor Performance Assessment Reporting System

CPFF – Cost Plus Fixed Fee

DOC – U.S. Department of Commerce

EVMS – Earned Value Management System

FAR – Federal Acquisition Regulations

NESDIS – National Environmental Satellite, Data, and Information Service

NOAA – National Oceanic and Atmospheric Administration

OSGS – Office of Satellite Ground Services (OSGS)

PM – Project Manager

PWS – Performance Work Statement

QASP – Quality Assurance Surveillance Plan

QCP – Quality Control Plan

RFIMS – Radio Frequency Interference Monitoring System

SOO – Statement of Objectives

Attachment 1: Quality Assurance Monitoring Form

RFIMS

Quality Assurance Monitoring Form

Section 1 – Quality Assurance Surveillance Activities

Applicable CLIN: Date of Review: Reviewer:

Title: Email: Phone:

Method of QA Monitoring: Surveillance ( ) Comms/Feedback ( )

Trend Analysis ( ) Project Review ( ) Other ( ) Describe:

Level of Surveillance: Monthly ( ) Quarterly ( ) As-Required ( ) 100% Inspection ( )

Service or Standard Requirement:

QASP AQL:

Contractor Performance:

Meets Standard: ( )

Does Not Meet Standard ( )

# of Times not Met:

Performance Rate (%): Deliverable Rejected: Y or N

Incentive or Disincentive Imposed:

Performance Narrative:

Section 2 – Contractor Deficiency Report

Type of Deficiency:

Deficiency Level: High ( ) Moderate ( ) Low ( )

How was Deficiency Level determined:

Risk Impact on program: High ( ) Moderate ( ) Low ( )

How was Risk Impact determined:

CAP Required: Y or N Status of CAP:

Under Development ( ) Approved ( ) Implemented ( )

CAP Implementation Timeframe:

CAP Strategy:

Responsible CAP Contractor Point of Contact:

Title:

Email:

Phone:

Date Reported to NOAA Contracting Officer: