Attachment 3 - QASP.pdf

QASP: NILE ISS CONTRACT

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

FOR THE

N0003918R0006

1. INTRODUCTION

This Quality Assurance Surveillance Plan (QASP) aligns to the requirements outlined in the NILE

In‐Service Support Performance Work Statement (PWS). This plan sets forth the procedures and guidelines that will be used to ensure required performance standards and services levels are met through the evaluation of Contractor performance.

1.2 Purpose

The purpose of the QASP is to describe the requirements and methods used to monitor performance and to identify the required documentation and the resources to be employed.

The QASP provides a means for evaluating whether the Contractor is meeting requirements identified in the PWS and to ensure that the Government receives best value services. This

QASP:

1) Defines the roles and responsibilities related to administration of the Task Order and associated QASP;

2) Identifies the performance objectives;

3) Defines the methodologies used to monitor and evaluate the Contractor’s performance, describes quality assurance documentation requirements; and

4) Describes the analysis of quality assurance monitoring results.

1.3 Performance Management Strategy

The Government’s quality assurance strategy will be implemented by specifically reviewing selected Contractor deliverables (CDRLs1). There will be two methods of surveillance that will be performed by the NILE PMO and the Contracting Officer Representative (COR) with respect to these specific deliverables. First, the COR will determine the timeliness of these deliverables, where applicable, as measured by the date upon which the Government receives the

Contractor deliverable. Second, the COR will measure the quality of the deliverable based on the outcome of the deliverables review process. See ATTACHMENT 1: PERFORMANCE

REQUIREMENTS SUMMARY.

2. ROLES AND RESPONSIBILITIES

2.1 Procuring Contracting Officer (PCO)

The PCO is responsible for monitoring contract compliance, contract administration, and cost control and for resolving any differences between the observations documented by the COR and responses from the Contractor.

1 Contract Data Requirements List

2.2 Contracting Officer Representative (COR)

The COR, designated in writing by the PCO, acts as the PCO's authorized representative to assist in administering the Task Order. COR functions and limitations are contained in the written appointment letter. The COR is responsible for technical administration of the programs and projects and ensures proper Government surveillance of the Contractor’s performance. While the COR may receive inputs and comments from Government functional leads regarding

Contractor performance, only the COR is authorized to officially inspect and accept work under the task order.

3. PERFORMANCE STANDARDS AND QUALITY LEVELS

The required performance standards and quality levels are included in the PWS, TDLs and in the

QASP, ATTACHMENT 1.

3.1 Standard for All Deliverables

Unless otherwise specified, all deliverables shall:

a) Be in accordance with content, format, due date, and distribution requirements;

b) Be delivered on or ahead of schedule in accordance with the PWS, CDRLs, and TDLs;

c) Be complete, clear, concise, and technically accurate;

d) Not require clarification or rework;

e) Be within projected costs; and

f) Be delivered in accordance with applicable references listed in Paragraph 4 of the PWS.

3.2 Performance Metrics

Work will be assessed against the contract, task order, and specific references therein.

Evaluations will be based on the Contractor’s:

a) Timeliness of Deliverable, where applicable.

b) Quality of Deliverable

4. METHODOLOGIES TO MONITOR PERFORMANCE

4.1 Surveillance Techniques

The primary methods of surveillance are:

1) Monitoring receipt date of deliverables for the purposes of determining the timeliness of the deliverables.

2) Administering the deliverables Review Process to facilitate the Government’s review and acceptance decisions for the purposes of determining the quality of the deliverables.

4.2 Customer Feedback

The Contractor is expected to establish and maintain professional communication between its employees and customers. The primary objective of this communication is customer satisfaction. Customer complaints must set forth clearly and in writing, the detailed nature of the complaint, must be signed, and must be forwarded to the COR. The COR will accept customer complaints, investigate and involve the PCO and Contractor where appropriate.

5. QUALITY ASSURANCE DOCUMENTATION

5.1 The Performance Management Feedback Loop

The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed in the PWS and TDLs, and monitored pursuant to ATTACHMENT 1.

5.2 Monitoring Forms

The Government’s surveillance, accomplished by the COR, will be collected using ATTACHMENT

2: SURVEILLANCE ACTIVITY TRACKER and reported in the metric charts depicted in

ATTACHMENT 1. Feedback will be given as actionable performance observations are noted.

The COR will retain a copy of all completed forms and quality assurance documentation.

6. ANALYSIS OF QUALITY ASSURANCE ASSESSMENT

6.1 Determining Performance

The Government will use the monitoring methods cited to determine whether the performance standards and service levels have been met. If the Contractor has not met the minimum requirements, the Contractor may be asked to develop a corrective action plan to show how and by what date it intends to bring performance up to the required levels.

6.2 Reporting

The COR will document and provide feedback as actionable performance observations are noted. On an annual basis, the COR will prepare a written report summarizing the overall results of the quality assurance surveillance of the Contractor’s performance. The report will enable the Government to demonstrate whether the Contractor is meeting the stated objectives and performance standards. Additionally, these reports will be retained for reference and may be input into the Contractors Performance Assessment Reporting System

(CPARS).

6.3 Reviews and Resolution

The Contractor will be required to meet with Government personnel as deemed necessary to discuss performance evaluation, to include In Process Reviews (IPR). The COR will define a frequency of in‐depth reviews with the Contractor, including appropriate self‐assessments by the Contractor; however, if the need arises, the Contractor will meet with the PCO, ACO, COR, PM and other Government customers as often as required. The COR will coordinate and communicate with the Contractor to resolve issues and concerns regarding marginal or unacceptable performance. The COR and the Contractor will jointly formulate tactical and long‐ term courses of action. Requests for changes to metrics, thresholds, or service levels should be clearly documented, supported with rationale. Changes to service levels, procedures, and metrics will be incorporated via a task order modification.

ATTACHMENT 1: PERFORMANCE REQUIREMENTS SUMMARY

QASP Metrics Quality assurance will be conducted by reviewing and managing the Contractor’s deliverables.

There will be two methods of surveillance that will be performed by the COR with respect to such deliverables. First, the COR will determine the timeliness of the deliverable, where applicable, as measured by the date upon which the Government receives the item. Second, the COR will measure the quality of the deliverable based on the outcome of the review process. The performance standard required with respect to the two surveillance components are described below:

CDRLs under Surveillance QASP metrics will be collected on the deliverables listed in the table below. Also described in the table is the “Date Due” for each deliverable, which will be used as the basis for making the

Timeliness determination.

CDRL Deliverables Date Due (NLT) PWS Section(s) A001 Project Management Plan (PMP) 30 calendar DACA 5.1.3.2

A002 Monthly Status Reports (MSRs) 15th calendar day of the month

5.1.3.3

A003 Test Plans/Procedures/Reports See CDRL 5.1.2.1.2, 5.1.2.1.7

A004 Configuration Management Information System (CMIS)

See CDRL 5.1.3.11.3.1

A005 Meetings (Post Award Conference, IPRs, IPTs, TRRs, CCRBs) Agendas

10 calendar days before event

5.1.3.5, 5.1.3.6, 5.1.3.7, 5.1.1.5.4, 5.1.3.11.3.2.1

A005 Meetings Minutes 5 calendar days after event 5.1.3.5, 5.1.3.6, 5.1.3.7, 5.1.1.5.4, 5.1.3.11.3.2.1

A006 Tech Documentation (CPs, PCRs, FRs, White Papers)

14 calendar days after completion of the analysis

As required, 5.1.1.1, 5.1.3.11.3.2.1

A008 SNC Software Test Report 15 calendar days after Tests

5.1.1.5, 5.1.1.5.2

Level 1 Surveillance: Timeliness Level 2 Surveillance: Quality

On‐Time >90% Approved without comments >75%

1‐5 Days Late <8% Approved after one Government Review <20%

>5 Days Late <2% Approved after two or more Government Reviews <5%

CDRL Deliverables Date Due (NLT) PWS Section(s)

A010 SNC Source and Executable Software (SES)

NLT 30 calendar days after each SNC SW Acceptance Test

5.1.1.2

A011 SNC/NRS BCR Technical Data Package (TDP), Including STMs and L22 Guidebook

30 calendar days after Tests

5.1.1, 5.1.1.2, 5.1.1.3, 5.1.1.3.1, 5.1.1.4, A013 NRS Software Test Report 15 calendar days after Tests

5.1.1.5.1, 5.1.1.5.2

CDRL

Deliverable

Type

Performance

Standard

Acceptable Quality Level

Approved without comments

Approved after one

Government Review

Approved after two or more Government

Reviews

A001 Project

Management

Plan (PMP)

PMP to address all requirements of PWS para

5.1.3.2

PMP adequately addresses all areas identified in para

5.1.3.2 of the PWS.

Government approves without comments.

PMP adequately addresses all areas identified in para

5.1.3.2 of the PWS.

Government comments are correctly incorporated following one

Government review.

PMP adequately addresses all areas identified in para

5.1.3.2 of the PWS.

Government comments are correctly incorporated following two or more

Government reviews.

A002 Monthly

Status

Reports

(MSRs)

MSR to address all requirements of PWS para

5.1.3.3 and TDL

MSR adequately addresses all areas identified in para

5.1.3.3 of the PWS and TDL requirements.

Government approves without comments.

MSR adequately addresses all areas identified in para

5.1.3.3 of the PWS and TDL requirements.

Government comments are correctly incorporated following one

Government review.

MSR adequately addresses all areas identified in para

5.1.3.3 of the PWS and TDL requirements.

Government comments are correctly incorporated following two or more

Government reviews.

A003 Test Plans/

Procedures/

Test Plans/

Procedures/

Test Plans/

Procedures/

Test Plans/

Procedures/Reports

Test Plans/

Procedures/Reports

Deliverable

Type

Performance

Standard

Acceptable Quality Level

Approved without comments

Approved after one

Government Review

Approved after two or more Government

Reviews

Reports Reports to address all PWS and TDL requirements of

PWS para

5.1.2.1.2, 5.1.2.1.7

Reports adequately address all areas identified in PWS and TDL.

Government approves without comments.

adequately address all areas identified in

PWS and TDL.

Government comments are correctly incorporated following one

Government review.

adequately address all areas identified in

PWS and TDL.

Government comments are correctly incorporated following two or more

Government reviews.

A004 Configuration

Management

Information

System

(CMIS)

CMIS to address all requirements of PWS para

5.1.3.11.3.1

CMIS adequately addresses all areas identified in para

5.1.3.11.3.1 of the

PWS and TDL requirements.

Government approves without comments.

CMIS adequately addresses all areas identified in para

5.1.3.11.3.1 of the

PWS and TDL requirements.

Government comments are correctly incorporated following one

Government review.

CMIS adequately addresses all areas identified in para

5.1.3.11.3.1 of the

PWS and TDL requirements.

Government comments are correctly incorporated following two or more

Government reviews.

A005 Meetings

(Post Award

Conference, IPRs, IPTs, TRRs, CCRBs)

Agendas

Agendas to address all requirements of

PWS para

5.1.3.5, 5.1.3.6, 5.1.3.7, 5.1.1.5.4, 5.1.3.11.3.2.1

Agendas adequately address all requirements of

PWS paras 5.1.3.5, 5.1.3.6, 5.1.3.7, 5.1.1.5.4, 5.1.3.11.3.2.1.

Government approves without comments.

Agendas adequately address all requirements of PWS paras 5.1.3.5, 5.1.3.6, 5.1.3.7, 5.1.1.5.4, 5.1.3.11.3.2.1.

Government comments are correctly incorporated following one

Government review.

Agendas adequately address all requirements of PWS paras 5.1.3.5, 5.1.3.6, 5.1.3.7, 5.1.1.5.4, 5.1.3.11.3.2.1.

Government comments are correctly incorporated following two or more

Government reviews.

A005 Meetings Meeting minutes Meeting minutes Meeting minutes to Meeting minutes

Deliverable

Type

Performance

Standard

Acceptable Quality Level

Approved without comments

Approved after one

Government Review

Approved after two or more Government

Reviews

(Post Award

Conference, IPRs, IPTs, TRRs, CCRBs)

Minutes to address all requirements of

PWS para

5.1.3.5, 5.1.3.6, 5.1.3.7, 5.1.3.8, 5.1.1.5.4, 5.1.3.11.3.2.1 adequately address all requirements of

PWS paras 5.1.3.5, 5.1.3.6, 5.1.3.7, 5.1.3.8, 5.1.1.5.4, 5.1.3.11.3.2.1.

address all requirements of PWS paras 5.1.3.5, 5.1.3.6, 5.1.3.7, 5.1.3.8, 5.1.1.5.4, 5.1.3.11.3.2.1 adequately address all requirements of PWS paras 5.1.3.5, 5.1.3.6, 5.1.3.7, 5.1.3.8, 5.1.1.5.4, 5.1.3.11.3.2.1.

A006 Technical

Documentatio n (WPs, CPs, PCRs, FRs, etc.)

Technical

Documentation

(WPs, CPs, PCRs, FRs, etc.) to address all PWS paras 5.1.1.1, 5.1.3.11.3.2.1 requirements and TDL requirements.

Technical

Documentation

(WPs, CPs, PCRs, FRs, etc.)

adequately addresses all areas identified in PWS and TDL.

Government approves without comments.

Technical

Documentation

(WPs, CPs, PCRs, FRs, etc.) adequately addresses all areas identified in PWS and

TDL. Government comments are correctly incorporated following one

Government review.

Technical

Documentation (WPs, CPs, PCRs, FRs, etc.)

adequately addresses all areas identified in

PWS and TDL.

Government comments are correctly incorporated following two or more

Government reviews.

A008 SNC Software

Test Report

(STR)

SNC STR to address all requirements of

PWS para

5.1.1.5, 5.1.1.5.2

SNC STR

adequately addresses all areas identified in PWS para 5.1.1.5 and

TDL. Government approves without comments.

SNC STR adequately addresses all areas identified in PWS para 5.1.1.5 and TDL.

Government comments are correctly incorporated following one

Government review.

SNC STR adequately addresses all areas identified in PWS para

5.1.1.5 and TDL.

Government comments are correctly incorporated following two or more

Government reviews.

A010 SNC Source and

Executable

Software

(SES)

SNC Source and

Executable

Software (SES) to address all

PWS para 5.1.1, SNC Source and

Executable

Software (SES) to address all PWS and TDL

SNC Source and

Executable Software

(SES) to address all

PWS and TDL requirements.

SNC Source and

Executable Software

(SES) to address all

PWS and TDL requirements.

Deliverable

Type

Performance

Standard

Acceptable Quality Level

Approved without comments

Approved after one

Government Review

Approved after two or more Government

Reviews

5.1.1.2, 5.1.1.3, 5.1.1.3.1, 5.1.1.4, and TDL requirements.

requirements.

Government approves without comments.

Government approves without comments.

Government comments are correctly incorporated following one

Government review.

Government approves without comments.

Government comments are correctly incorporated following two or more

Government reviews.

A011 SNC/NRS BCR

Technical

Data Package

(TDP),

Including

STMs and L22

Guidebook

SNC/NRS BCR

TDP (Including

STMs and L22

Guidebook) to address all PWS and TDL requirements.

SNC/NRS BCR TDP

(Including STMs and L22

Guidebook) adequately addresses all areas identified in PWS and TDL.

Government approves without comments.

SNC/NRS BCR TDP

(Including STMs and

L22 Guidebook) adequately addresses all areas identified in

PWS and TDL.

Government comments are correctly incorporated following one

Government review.

SNC/NRS BCR TDP

(Including STMs and

L22 Guidebook) adequately addresses all areas identified in

PWS and TDL.

Government comments are correctly incorporated following two or more

Government reviews.

A013 NRS Software

Test Report

(STR)

NRS STR to address all requirements of

PWS para

5.1.1.5.1, 5.1.1.5.2

NRS STR

adequately addresses all areas identified in PWS para 5.1.1.5 and

TDL. Government approves without comments.

NRS STR adequately addresses all areas identified in PWS para 5.1.1.5 and TDL.

NRS STR to address all requirements of PWS para 5.1.1.5

QASP Administration via CDRL Review Process Administration of the QASP will be conducted in concert with the CDRL Review Process. The outcomes of the process will serve to provide the data necessary for determining the QASP metrics.

QASP Metric Administration and Reporting The metrics derived from QASP administration will be conveyed within regular QASP reports, using chart displays which may be like shown below.

Level 1 Surveillance: Timeliness

Level 2 Surveillance: Quality

ATTACHMENT 2: SURVEILLANCE ACTIVITY TRACKER

The COR will maintain the QASP Activity Tracker to ensure accurate and continuous metric reporting. The tracker will provide the data needed to report on both the timeliness and quality of the Contractor’s deliverables.

CDRL # Description Due Date Date Received Initial Status Gov’t comments date Final Status

Level 1 Surveillance Level 2 Quality

CDRL# Page/Para Originator Comment Government Comments