Atch 8 DREN4 QASP Amend 0001.pdf

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

As of 17 January 2020

1. CONTRACT NUMBER & TITLE

Defense Research & Engineering Network 4 (DREN4) Contract

2. PURPOSE

This QASP provides a systematic method to conduct surveillance of the Contractor's performance for work performed under the stated contract. This QASP explains the following:

What will be monitored?

How monitoring will take place.

Who will conduct the monitoring?

How monitoring efforts and results will be documented.

This QASP does not detail how the Contractor accomplishes the work. Rather, the QASP is created with the premise that the Contractor is responsible for management and quality control actions to meet the terms of the contract. It is the Government's responsibility to be objective, fair, and consistent in evaluating performance. In addition, the QASP recognizes that unforeseen and uncontrollable situations may occur.

This QASP is a “living document” and the Government may review and revise it on a regular basis. Updates will ensure that the QASP remains a valid, useful, and enforceable document in synchronization with the PWS as it may change.

3. ROLES AND RESPONSIBILITIES

The following personnel will oversee and coordinate surveillance activities.

a) Program/Project Manager (PM)

The PM provides program oversight and supports the COR's performance assessment activities.

b) Contracting Officer (KO)

The KO will ensure performance of all necessary actions for effective contracting, ensure compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The KO will also ensure that the Contractor receives impartial, fair, and equitable treatment under this contract and determine the final assessment of the Contractor's performance.

c) Contracting Officer's Representative (COR)

The COR is responsible for providing continuous technical oversight of the Contractor's performance.

The COR uses the QASP to guide the oversight/surveillance process. The COR will keep Quality

Assurance records that accurately document the Contractor's actual performance. The purpose of the Surveillance Plan is to monitor that the Contractor meets the performance standards contained in the contract. The COR will report early identification of performance problems to the KO. The COR is not empowered to make any contractual commitments or to authorize any contractual change on the

Government's behalf.

4. METHODS OF SURVEILLANCE

This QASP guides monitoring performance as stated in the PWS primarily using Periodic Inspection. Some

Performance Surveillance will be performed also using the 100 Percent Inspection and Random Sampling methods.

5. PERFORMANCE STANDARDS

Task Performance

Standard

Acceptable Quality Level (AQL) Surveillance Method Incentives (+/-)

6.0.2 General

Guidance

Deliverable schedule and accuracy

Deliverables and all revisions and corrections to deliverables are on time, accurate, and error free.

Periodic Inspection:

Government review of

Deliverables for delivery time, accuracy, and errors.

-.5% of NOC MRC per incident of not meeting any of the delivery time, accuracy, and error requirements.

-.5% of NOC MRC for each additional 7 days past the delivery time requirement.

6.1.3.1

Latency

Latency

Formula

For all DREN Nodes, no more than 5 pairs exceed Latency

Formula requirements.

Periodic Inspection:

Government review of contractor developed

Latency Report.

See Node Credits (6.5.7.1.2 and 6.5.7.2.2)

6.1.3.2 Jitter Jitter

Measurement

Jitter between all pairs of DREN

Nodes is less than 1 milliseconds.

Random Sampling:

Government review of jitter as reported in the 6.5.9.1.3

Data Dashboard

See Node Credits (6.5.7.1.2 and 6.5.7.2.2)

6.1.3.3 Loss Loss

Measurement

Loss on all DREN paths is less than 0.01%.

Random Sampling:

Government review of loss as reported in the 6.5.9.1.3

Data Dashboard

See Node Credits (6.5.7.1.2 and 6.5.7.2.2)

6.4.8

Combating

Trafficking in

Persons

Compliance with FAR

52.222-50

The Contractor immediately reports issues to KO

Periodic Inspection:

Program Management

Review (PMR) infrequent compliance issues noted and annual inspection

None

6.5.1 Service

Provisioning

Service

Provisioning

Elements a) through f)

The Contractor delivers service as required from a Government issued TSO and within the timelines specified in Table 6.12

Required Provisioning Intervals for DITCO-issued service orders.

Periodic Inspection:

Government review of

Service Request

Provisioning Performance

Weekly Report and individual 6.5.13.2

Acceptance Test Reports.

New Node or upgrade timeline

NRC

Plus/Minus

On time No +/-

1-10 days late -2%

11-20 days late -4%

21-30 days late -8%

> 31 days late -16%

6.5.2 Change

Request

Provisioning

Change

Request

Provisioning

The Contractor provisions change requests on time and

100% accurate/error free.

100 Percent Inspection:

Government validation of change requests and review of Change Request

Provisioning Performance

Report.

-1% of NOC MRC per incident of not meeting 100% accurate and on time.

6.5.4

Contractor

Managed

Network

Database

Contractor

Managed

Network

Database

The Contractor maintains a

Network Database that is 100% accurate/error free.

All errors detected by the

Contractor, by the validation process or reported by the

Government are corrected within seven days.

Periodic Inspection:

Government review of

Validation report and

Random Sampling:

Spot checks of Network

Database.

-.5% of NOC MRC if greater than 5% errors are identified in the month.

6.5.6 Problem

Management

Problem

Resolution /

Management

The Contractor meets all requirements specified in this subtask on time and error free

Periodic Inspection:

Government review of

Problem Management

Reports

-1% of NOC Monthly Recurring fee for each incident that does not meet the required times.

In addition for DREN Nodes with the Rapid Response option:

-100% of DREN Nodes Rapid

Response Option fee when one or more incidents occur in a month on that DREN Node that does not meet the required times.

6.5.7.1 DREN

Node Outages

6.5.7.1.1

Node Outage

Time minimal Periodic Inspection:

Government review of

DREN Node and Network

Availability Report

Outage Node MRC

Plus/Minus

<15 minutes

0%

>=15 minutes

Credit calculated per 6.5.7.1.2

DREN Node MRC

Billing Credits

6.5.7.2

Network

Availability

ANA The Contractor maintains an

ANA of 99.9% or greater.

Periodic Inspection:

Government review of

Network Availability Report

Availability MRC Plus/Minus

>=99.9% 0%

<99.9% Credit calculated per

6.5.7.2.2 ANA

Total MRC

Billing Credits

6.5.10 OOB

Access

OOB Access

Function

The Contractor deploys and maintains an OOB Access capability, performs OOB Access

Testing weekly and repair failures within a week

Periodic Inspection:

Government review of OOB

Status Reports

None

6.5.14 DNS

Service

Accurate DNS

Resource

Records

The Contractor immediately updates records to reflect changes and continuously audits the DNS records and maintains 100% accuracy

Periodic Inspection:

Monthly report (6.4.1.8) of changes and auditing of

DNS records missing or inaccurate

70% to<90% -2% NOC MRC

<70% -10% NOC MRC

6. DOCUMENTING PERFORMANCE

The Government will use the Deliverables from the PWS in the following table as part of the documentation of

Contractor performance.

PWS

Task#

Deliverable

Number/Title

Format Due Date Distribution/Copies Frequency and

Remarks

6.1.3.5 1: DCN Placement

Current Map(s)

Contractor-

Determined Format

15 days after

Government request

Standard Distribution

(6.0.2)

Monthly through

CITP, Annually after

CITP, after any significant changes, and upon

Government Request

6.1.3.5 2: DCN Placement

Current Report

Contractor-

Determined Format

15 days after

Government request

Standard Distribution

(6.0.2)

Monthly through

CITP, Annually after

CITP, after any significant changes, and upon

Government Request

6.1.3.5 3: DCN Placement

Proposed Map(s)

Contractor-

Determined Format

15 days after

Government request

Standard Distribution

(6.0.2)

Monthly through

CITP, Annually after

CITP, after any significant changes, and upon

Government Request

6.1.3.5 4: DCN Placement

Proposed Report

Contractor-

Determined Format

15 days after

Government request

Standard Distribution

(6.0.2)

Monthly through

CITP, Annually after

CITP, after any significant changes, and upon

Government Request

6.1.3.6.1 5: Latency Report Contractor-

Determined Format

15th of each month Standard Distribution

(6.0.2)

Monthly

6.1.3.6.2 6: On Demand

Pairwise TCP Test

Report

Contractor-

Determined Format

15 days after

Government request

Standard Distribution

(6.0.2)

Upon Government

Request

6.1.3.6.3 7: On Demand

Aggregate TCP Test

Report

Contractor-

Determined Format

15 days after

Government request

Standard Distribution

(6.0.2)

Upon Government

Request

6.1.3.6.4 8: On Demand

Pairwise Throughput

Test Report

Contractor-

Determined Format

15 days after

Government request

Standard Distribution

(6.0.2)

Upon Government

Request

6.1.3.6.5 9: On Demand

Aggregate

Throughput Test

Report

Contractor-

Determined Format

15 days after

Government request

Standard Distribution

(6.0.2)

Upon Government

Request

6.3.1 10: RMF Artifacts Contractor-

Determined Format

Within 15 days after any change to the physical, logical, or flow aspects of the network or other security impacting changes eMASS filings; RADIX uploads; ACAS filings;

and other CSSP uploading and filing requirements

Every time there are significant changes

6.3.1.1 11: RMF IATT

Artifacts

Government provided templates to be provided after award

220 days after contract award eMASS filings; RADIX uploads; ACAS filings;

and other CSSP uploading and filing requirements

One Time

6.3.1.2 12: RMF ATO

Artifacts

Government provided templates to be provided after award

243 days after contract award eMASS filings; RADIX uploads; ACAS filings;

and other CSSP uploading and filing requirements

One Time

6.3.1.3 13: Control Review

Report

Contractor determined format

243 days after contract award eMASS filings; RADIX uploads; ACAS filings;

and other CSSP uploading and filing requirements

One Time

6.3.1.3 14: POA&M Status

Report

Contractor determined format

15th of each month Standard Distribution

(6.0.2) plus the DREN

ISSM

Monthly

6.3.1.3 15: Physical, Logical, and Flow Network

Diagrams

Contractor determined format

Within 15 days of every 180 day revalidation

Upload to the

Government

Documentation

Repository (draft and final) and to eMASS

(final)

Every 180 days

6.3.1.3 16: Resubmitted

RMF Artifacts

Contractor determined format

Within 15 days after any change to the physical, logical, or flow aspects of the network or other security impacting changes, and every

SCA-V assessment eMASS filings; RADIX uploads; ACAS filings;

and other CSSP uploading and filing requirements

Every time there are significant changes or a SCA-V assessment

6.3.2 17: CCB Change

Request

Contractor determined format

30 days prior to requested change date

Standard Distribution

(6.0.2)

As required

6.3.8 18: SDREN CAP

documentation

Government provided format after contract award

No more than 180 days after commencement of

Phase II of transition

Standard Distribution

(6.0.2)

One time, update as required

6.4.1.1 19: Program Task

Management Report

Contractor-

Determined Format

(includes presentation, minutes and associated materials)

15th of each month Standard Distribution

(6.0.2)

Monthly

6.4.1.3 20: Project

Management

Planning and Control

Report

Contractor

Determined Format

30 days after

Contract Award and seven days after any substantive changes occur

Standard Distribution

(6.0.2)

Any time there is a substantive change

6.4.1.5 21: Progress Report Contractor

Determined Format

(risk management, resources, task completion, technical support hours, significant activities, accomplishments)

15th of each month Standard Distribution

(6.0.2)

Monthly

6.4.1.6.1 22: TIM

Presentations

Government provided prior to

TIM

21 days prior to the

TIM

Standard Distribution

(6.0.2) and presented at the meeting

Annually

6.4.1.6.1 23: TIM Minutes Contractor

Determined Format

Seven days after the

TIM

Standard Distribution

(6.0.2)

Annually

6.4.1.8 24: PMR Report Contractor

Determined Format

(presentations, minutes, and associated materials)

15th of each month Standard Distribution

(6.0.2)

Monthly

6.4.1.9 25: Annual Planning and Design Review

Report

Contractor

Determined Format

(presentations, minutes, and associated materials)

Presentations 21 days prior to review meeting, minutes

Seven days after meeting

Standard Distribution

(6.0.2)

Annually

6.4.2 26: Network Status

Report

Contractor

Determined Format

(presentations, minutes, and associated materials)

15th of each month Standard Distribution

(6.0.2)

Monthly

6.4.3.1 27: TIP Contractor

Determined Format

14 days after

Government request or as needed for

Contractor initiated

Standard Distribution

(6.0.2)

As required

6.4.5 28: Accounting

Management Report

Contractor

Determined Format

15th of each month Standard Distribution

(6.0.2)

Monthly

6.4.7 29: SCRM Plan Contractor-

Determined Format no more than 25 pages

Draft: 30 days after award

Final: 30 days after receipt of

Government feedback

Update: five days after any change

Standard Distribution

(6.0.2)

One time, update as required

6.5.1 30: Service Request

Provisioning

Performance weekly report

Contractor-

Determined Format

Weekly by each

Thursday at 12:00, Eastern Time

Standard Distribution

(6.0.2)

Weekly

6.5.2 31: Change Request

Provisioning

Performance Report

Contractor-

Determined Format

15th of each month Standard Distribution

(6.0.2)

Monthly

6.5.4 32: Validation

Report

Contractor-

Determined Format

15th of each month Standard Distribution

(6.0.2)

Monthly

6.5.5 33: Response Metric

Report

Contractor-

Determined Format

15th of each month Standard Distribution

(6.0.2)

Monthly

6.5.5 34: Help Desk SOP Contractor-

Determined Format

Prior to IPC

Demonstration

Standard Distribution

(6.0.2)

Full document delivered completely once, changes reviewed with

Government after

6.5.6 35: Problem

Management Report

Contractor-

Determined Format

15th of each month Standard Distribution

(6.0.2)

Monthly

6.5.7.3 36: Network

Availability Report

Contractor-

Determined Format

15th of each month Standard Distribution

(6.0.2)

Monthly

6.5.8 37: Problem

Escalation Procedure

Contractor-

Determined Format

Prior to IPC

Demonstration

Standard Distribution

(6.0.2)

One time, changes reviewed annually

6.5.9.1.1 38: API

Documentation

Contractor

Determined Format

Beginning of IPC

Demonstration

Standard Distribution

(6.0.2)

One time, updated within 15 days of API changes

6.5.10 39: OOB Status

Report

Microsoft Office

Excel spreadsheet with Contractor determined layout

Weekly by each

Thursday at 12:00, Eastern Time

Standard Distribution

(6.0.2) weekly

6.5.11 40: Regression

Testing Plan

Contractor-

Determined Format

15 days after regression testing ticket opened

Standard Distribution

(6.0.2)

As required

6.5.11 41: Regression Test

Report

Contractor-

Determined Format

Seven days after regression testing is complete

Standard Distribution

As required

6.5.11 42: Change/upgrade

MOP

Contractor-

Determined Format

Seven days after regression testing is complete

Standard Distribution

(6.0.2)

As required

6.5.11 43: Upgrade

Schedule

Contractor-

Determined Format

Seven days after regression testing is complete

Standard Distribution

(6.0.2)

As required

6.5.12.1 44: DREN Node

Master Plan

DREN Node Master

Plan Template

Draft version 90 days after contract award;

final version 15 days after receiving comments and recommendations from the

Government

Standard Distribution

(6.0.2)

One time, and updated as necessary after

6.5.12.2 45: Individual DREN

Node As-Built Report

Contractor-

Determined Format

Seven days after

Government acceptance of a new or upgraded DREN

Node

Standard Distribution

(6.0.2)

As required

6.5.12.3 46: DREN Node Site

Survey Template

Contractor-

Determined Format

Draft: 21 days after contract award

Final: 21 days after

Government feedback on draft

Update: within seven days of any significant change

Standard Distribution

(6.0.2)

One time, update as required

6.5.12.4 47: DREN Node Site

Survey

Contractor-

Determined Format

30 + seven days after

DITCO issues service order for new/change

21 + seven days after

DITCO issues service order for termination

Standard Distribution

(6.0.2)

As required

6.5.12.5 48: DREN Node

Location

Modification Plan

Contractor-

Determined Format

14 days after delivery of DREN

Node Site Survey

Standard Distribution

(6.0.2)

As required

6.5.13.1 49: Acceptance Test

Plan

Contractor-

Determined Format

Prior to IPC

Demonstration

Standard Distribution

(6.0.2)

One time, update as required

6.5.13.2 50: Acceptance Test

Report

Contractor-

Determined Format

Seven days following satisfactory testing

Standard Distribution

(6.0.2)

As required

6.6.1.1.1 51: IPC Plan Contractor-

Determined Format

Draft version 60 days after contract award;

final version 80 days after contract award

Standard Distribution

(6.0.2)

One time

6.6.1.1.2 52: RTM Template provided by the Government following contract award

60 days after contract award

Standard Distribution

(6.0.2)

One time, annually, and as required

6.6.1.1.4 53: IPCDP Contractor-

Determined Format

Draft version 120 days after IPC Plan submission

Final version 140 days after final IPC

Plan submission

Standard Distribution

(6.0.2)

One time

6.6.1.1.6 54: IPCDR Contractor-

Determined Format

Seven days after completion of the IPC

Test and

Demonstration

Standard Distribution

(6.0.2)

One time

6.6.1.3 55: CITP Contractor-

Determined Format

200 days after contract award

Standard Distribution

(6.0.2)

One time, updated as required

6.6.3.1.1 56: Development of

Contract and

Network Phase-Out

Transition Plan

Contractor-

Determined Format

60 calendar days after award of follow-on DREN contract

Standard Distribution

One time

The Government will document the two types of performance:

a) Acceptable Performance

Positive Performance will be documented in the COR Report and significant positive performance also as a Contractor Performance Assessment Reporting System (CPARS) entry.

b) Unacceptable performance

Unacceptable Contractor performance will be identified from surveillance evaluations. When unacceptable performance occurs, the COR will inform the Contractor. The COR will document the form and content of the discussion and place it in the COR Report which will be reviewed by the KO.

When the COR/KO determines formal written communication is required, the COR will prepare a

Corrective Action Report (CAR). The CAR is forwarded to the KO for dissemination to the Contractor.

The Contractor shall acknowledge receipt of the CAR in writing. The CAR will specify if the Contractor is required to prepare a Corrective Action Plan (CAP) to document how the Contractor shall correct the unacceptable performance and avoid any recurrence. The CAR will also state how long after receipt the Contractor has to present this CAP to the KO and COR. The Government may withhold payment for services until corrective action is taken. The Government will review the Contractor's

CAP to determine acceptability. Upon Government acceptance of the corrected services the

Contractor will be permitted to invoice.

Any CAR may become a part of the supporting documentation for contract payment deductions, fee

(if any) deductions, CPARS entries, or other actions deemed necessary by the KO.

7. MANDATORY OR REGULATORY COMPLIANCE ITEMS

The Contractor shall provide initial proof of compliance with regulatory and compliance items contained in the contract. The COR and the KO will evaluate the initial submission and will conduct ad hoc reviews of the

Contractor's compliance throughout the remainder of the contract. The Contractor shall monitor compliance as required by FAR 52.222-50, Combating Trafficking in Persons, and report any issues to the KO.

8. ACCEPTABLE QUALITY LEVEL (AQL)

The AQLs for this contract are included in the table in 5. Performance Standards

9. EVALUATION METHOD

The COR will document receipt of all Deliverables as well as any verbal or electronic communications regarding performance on the contract including the times of delivery. The COR and as necessary the KO will review this information to perform evaluation of contract performance.

10. PERFORMANCE RATING

In evaluating the quality of Contractor's performance, the following performance ratings may be used (same as CPAR ratings).

Performance

Rating Criteria

Exceptional Performance meets contractual requirements and exceeds many to the government’s benefits. The contractual performance of the element or sub-element being assessed was accomplished with no problems and Contractor actions were highly effective.

Very Good Performance meets contractual requirements and exceeds some to the government benefits. The contractual performance was accomplished with few minor problems for which corrective actions taken by the Contractor were effective.

Satisfactory Performance meets contractual requirements. The contractual performance contains some minor problems for which corrective actions taken by the Contractor were satisfactory.

Marginal Performance does not meet some contractual requirements. The contractual performance reflects a serious problem for which the Contractor has not yet identified corrective actions. The Contractor’s proposed actions were marginally effective or were not completely implemented.

Unsatisfactory Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance contains serious problem(s) for which the Contractor’s corrective actions appear to have been or were ineffective.

APPENDIX A. PERFORMANCE ASSESSMENT REPORT

PERFORMANCE ASSESSMENT REPORT (PAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACT/TASK ORDER

NUMBER

2. CONTRACTOR 3. TYPE OF SERVICES

4. QUALITY ASSURANCE PERSONNEL (COR) SIGNATURE AND DATE

5. COR PHONE 6. SUSPENSE DATE

I. PERFORMANCE

7. DEFICIENCY (CHECK ALL BOXES THAT

APPLY)

NEW

REPEAT

NO DEFICIENCY NOTED

8. SERVICES SUMMARY or PWS PARAGRAPH ITEM

REVIEWED

9. BRIEF DESCRIPTION OF DEFICIENCY (IF DEFICIENCY

BOX WAS CHECKED)

10. DETAILED PERFORMANCE ASSESSMENT

II. CONTRACTOR VALIDATION

11. CONTRACTOR REPRESENTATIVE CONCUR

NON-CONCUR

12. CORRECTIVE ACTION ESTIMATED COMPLETION

DATE

13. CONTRACTOR REPRESENTATIVE CORRECTIVE ACTION AND PREVENTION OF RECURRENCE OR REASON

FOR NON-CONCURRENCE OF COR CITED DEFICIENCY

III. ACTION CORRECTED

14. CONCUR NON-CONCUR COR SIGNATURE AND DATE

15. COR REMARKS (REQUIRED)

6. CONTRACTOR REPRESENTATIVE REMARKS

APPENDIX B. CORRECTIVE ACTION REPORT

CORRECTIVE ACTION REPORT (CAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACTOR

2. CONTRACT NUMBER

3. TYPE OF SERVICES

4. FUNCTIONAL AREA

5. SUSPENSE DATE

6. CONTROL NUMBER

7. DEFICIENCY MAJOR MINOR

FINDING:

FINDING IMPACT:

Please respond with a written corrective action plan that details the corrective action of the cited deficiency, the cause of the deficiency, and actions taken to prevent recurrence by Suspense Date in Block 5. If date was not entered in Block 5, the contractor is not required to provide a response.

8. QUALITY ASSURANCE PERSONNEL (COR)

TYPED NAME AND GRADE

SIGNATURE AND DATE

9. ISSUING AUTHORITY

TYPED NAME AND GRADE

SIGNATURE AND DATE

10. COR RESPONSE TO CONTRACTOR CORRECTIVE ACTION AND ACTION TAKEN TO PREVENT RECURRENCE

11. COR DETERMINATION

ACCEPTED REJECTED

12. CLOSE DATE