22-R-7701_CVR_RFP Attachment 6-QASP v2.5 (sam.gov).pdf

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

22 July 2022

1. Contract or Task Order Title: Cyber Vulnerabilities Research (CVR)

2. Purpose:

The CVR QASP provides a systematic method to evaluate performance for the stated contract. This QASP explains the following:

What will be monitored?

How monitoring will take place.

Who will conduct the monitoring?

How monitoring efforts and results will be documented.

This QASP does not detail how the contractor accomplishes the work. Rather, the QASP is created with the premise that the contractor is responsible for management and quality control actions to meet the terms of the contract. It is the Government’s responsibility to be objective, fair, and consistent in evaluating performance. In addition, the

QASP should recognize that unforeseen and uncontrollable situations may occur.

This QASP is a “living document” and the Government may review and revise it on a regular basis. Updates shall ensure that the QASP remains a valid, useful, and enforceable document. Copies of the original QASP and revisions shall be provided to the contractor and Government officials implementing surveillance activities.

3. Roles and Responsibilities:

The following personnel shall oversee and coordinate surveillance activities.

Product Manager (PdM) – The PM provides program oversight and supports the COR’s performance assessment activities.

Assigned PM: Dave Pasquill

Organization or Agency: PM MCCO

Telephone: 443-654-1969

Email: djpasqu@nsa.gov

Contracting Officer (KO) – The KO shall ensure performance of all necessary actions for effective contracting, ensure compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The KO shall also ensure that the contractor receives impartial, fair, and equitable treatment under this contract. Determine the final assessment of the contractor’s performance.

Assigned KO: Michael Jackson

Organization or Agency: MARFORSYSCOM

Telephone: 443-654-0968

Email: michael.d.jackson1@usmc.mil

Contracting Officer’s Representative (COR) – The COR is responsible for providing continuous technical over-sight of the contractor’s performance. The COR uses the QASP to conduct the oversight/surveillance process. The

COR shall keep a Quality Assurance file that accurately documents the contractor’s actual performance. The pur-pose is to ensure that the contractor meets the performance standards contained in the contract. The COR is respon-sible for reporting early identification of performance problems to the KO. The COR is required to provide an an-mailto:djpasqu@nsa.gov mailto:michael.d.jackson1@usmc.mil nual performance assessment to the KO which will be used in documenting past performance. The QASP is the pri-mary tool for documenting contractor performance. The COR is not empowered to make any contractual commit-ments or to authorize any contractual change on the Government’s behalf.

Assigned COR: Torrence D. Moore

Organization or Agency: PM MCCO

Telephone: 443-654-3256

Email: tdmoor2@nsa.gov

G-9 (User Representative) - The G-9 shall receive, verify and validate that the product meets the user require-ment(s). Once product is acceptable to the users, the G-9 shall inform the COR through written correspondence, the product received met requirement(s).

Assigned User Representative: LtCol Plot, Joseph

Organization or Agency: Marine Corps Forces Cyberspace Command

Telephone: 443-654-0237

Email: japlot@nsa.gov

4. Primary Method of Surveillance:

100 Percent Inspection. This is usually only the most appropriate method for infrequent tasks or tasks with stringent performance requirements, e.g., where safety or health is a concern. With this method, per-formance is inspected/evaluated at each occurrence. One hundred percent inspection is too expensive to be used in most cases.

Customer Input. Although usually not a primary method, this is a valuable supplement to more system-atic methods. For example, in a case where random sampling indicates unsatisfactory service, customer complaints can be used as substantiating evidence. In certain situations where customers can be relied upon to complain consistently when the quality of performance is poor (e.g., dining facilities or building services), customer surveys and customer complaints may be a primary surveillance method, and customer satisfaction an appropriate performance standard. In all cases, complaints should be documented, prefera-bly on a standard form.

Surveillance Matrix

The Surveillance Matrix (Appendix 1) is the list of performance objectives and standards that must be performed by the contractor. This matrix details the method of surveillance the COR will use to validate and inspect these perfor-mance elements. Inspection of each element will be documented in the COR file.

Performance objectives define the desired outcomes. Performance Standards define the level of service required under the contract to successfully meet the performance objective. The inspection methodology defines how, when, and what will be assessed in measuring performance. The Government performance surveillance, using this QASP, to determine the quality of the contractor’s performance as it relates to the performance element standards. The Ac-ceptable Quality Levels (AQL) identified in the Surveillance Matrix should be used to form the foundation of the

COR’s inspection checklist.

5. Performance Standards:

Provide services and products to develop, test, deliver and employ exploits, payloads, and implants

Quality Control

Quality Assurance

Security Requirements Compliance

Physical Control mailto:tdmoor2@nsa.gov mailto:japlot@nsa.gov

6. Documenting Performance:

Acceptable Performance.

The Government shall document positive performance. A Performance Assessment Report (PAR) template is at-tached (Appendix 2). Any report may become part of the supporting documentation for fixed price payments, award/incentive fee payments, Contractor Performance Assessment Reporting System (CPARS) entries, or other actions. Government acceptance of services permits the contractor to invoice for the service fee.

Unacceptable performance.

Unacceptable contractor performance can be identified by customer complaints (Appendix 3) or upon schedule sur-veillance evaluations. When unacceptable performance occurs, the COR shall inform the contractor. This will nor-mally be in writing unless circumstances necessitate verbal communication. The COR shall document the discus-sion and place it in the COR file and provide a copy to the KO.

When the COR determines formal written communication is required, the COR shall prepare a Corrective Action

Report (CAR). The CAR is forwarded to the KO for dissemination to the contractor. A CAR template is attached to this QASP (Appendix 4).

The contractor shall acknowledge receipt of the CAR in writing. The CAR will specify if the contractor is required to prepare a corrective action plan to document how the contractor shall correct the unacceptable performance and avoid a recurrence. The CAR will also state how long after receipt the contractor has to present this corrective ac-tion plan to the KO and COR. The Government may withhold payment for services until corrective action is taken.

The Government shall review the contractor's corrective action plan to determine acceptability. Upon Government acceptance of the corrected services the contractor will be permitted to invoice for the service fee.

Any CAR may become a part of the supporting documentation for contract payment deductions, fixed fee deduc-tions, award fee nonpayment, CPARS or other actions deemed necessary by the KO.

7. Mandatory or regulatory compliance items:

The Contractor will provide initial proof of compliance with regulatory and compliance items contained in the con-tract. The COR and the KO will evaluate the initial submission and will conduct ad hoc reviews of the contractor's compliance throughout the remainder of the contract. The Contractor will monitor compliance as required by FAR

52.222-50, Combating Trafficking in Persons, and report any issues to the KO. See Combating Trafficking in Per-sons (CTIP) Homepage.

8. Acceptable Quality Level (AQL):

The AQL for this project is 100% due to the critical mission importance required during cyber operations.

9. Evaluation Method:

The Users’ will inspect, verify and validate products meet requirements. The COR will receive written confirmation from the User community that product is satisfactory.

10. Performance Rating:

In evaluating the quality of Contractor’s performance, the following performance ratings may be used (same as

CPAR ratings).

Performance

Rating Criteria https://ctip.defense.gov/ https://ctip.defense.gov/

Exceptional

Performance meets contractual requirements and exceeds many to the govern-ment’s benefits. The contractual performance of the element or sub-element being assessed was accomplished with no problems and contractor actions were highly ef-fective.

Very Good

Performance meets contractual requirements and exceeds some to the government benefits. The contractual performance was accomplished with few minor problems for which corrective actions taken by the contractor were effective.

Satisfactory

Performance meets contractual requirements. The contractual performance con-tains some minor problems for which corrective actions taken by the contractor were satisfactory.

Marginal

Performance does not meet some contractual requirements. The contractual perfor-mance reflects a serious problem for which the contractor has not yet identified cor-rective actions. The contractor’s proposed actions appear marginally effective or were not fully implemented.

Unsatisfactory

Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance contains serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

The Surveillance Matrix (Appendix 1) is the list of performance objectives and standards that must be performed by the contractor. This matrix details the method of surveillance and frequency the COR will use to validate and inspect these performance elements. Inspection of each element will be documented in the COR file.

Appendix 1 - Surveillance Matrix

Performance Ob-jectives

PWS Refer-ence(s) Performance Standard

Surveillance

Type

Acceptable Quality

Levels

Methods of

Calculation

Provide services and products to develop, test, de-liver and employ exploits, pay-loads, and im-plants

1.4

5.2

Standards:

- Visibility into target space to assess patterns in the tech-nological, physical and per-sona domains.

- Understanding of inherent vulnerabilities, in these do-mains to identify advantage or capability gaps.

- Competent planning to align advantages or capability gaps with operational plan-ning objectives.

- Ability to match capability development, to close capa-bility gaps and exploit ad-vantage.

- Ability to apply developed capabilities at will and at speed, to meet objectives.

Customer In-put

Contractor shall de-liver services or prod-ucts capable of meet-ing operational re-quirements 100% of the time

User representative will inform COR in writing that product or service meet op-erational need

Threshold: Contractor will develop or purchase exploits, payloads and implants that meet user requirements

Quality Control 1.6.1

Standard: The contractor shall develop and maintain an ef-fective quality control pro-gram to ensure services are performed in accordance with this PWS.

Threshold: The contractor shall submit an updated QCP within 5 working days when changes are made thereafter.

100% Inspec-tion

Quality control pro-gram will meet all regulatory standards

Visual review

Quality Assur-ance 1.6.2

Standard: The Contractor shall perform in accordance with the performance stand-ards.

Threshold: The Government shall evaluate the Contrac-tor’s performance under this contract in accordance with the Quality Assurance Sur-veillance Plan.

100% Inspec-tion

Customer In-put

Contractor shall ad-here to the Quality

Assurance Plan dur-ing all period of per-formances.

COR contacting user representative to determine cus-tomer satisfaction.

Security Re-quirement Com-pliance

1.6.7

Standards: The contractor shall be in 100% compliance of all security requirements defined in this PWS immedi-ately upon contract start of work and shall maintain

100% compliance throughout the contract.

cu:

Threshold: Contractor shall immediately report any secu-rity non-compliance to the

Government

100% Inspec-tion 100% Compliance

Contractor shall pro-vide COR evidence of 100% compliance in monthly report.

Physical Security 1.6.7.1

Standards: Contractor shall be responsible for safeguarding all government equipment, in-formation and property pro-vided for contractor use.

Threshold: Contractor will supply Government with In-ventory review annually.

100 % Inspec-tion 100% Compliance Visual review

Appendix 2 – Performance Assessment Report

PERFORMANCE ASSESSMENT REPORT (PAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACT/TASK ORDER

NUMBER

2. CONTRACTOR 3. TYPE OF SERVICES

4. QUALITY ASSURANCE PERSONNEL (COR) SIGNATURE AND DATE

5. COR PHONE 6. SUSPENSE DATE

I. PERFORMANCE

7. (CHECK ALL BOXES THAT APPLY)

DEFICIENCY

NEW

REPEAT

NO DEFICIENCY NOTED

8. SERVICES SUMMARY or PWS PARAGRAPH ITEM

REVIEWED

9. BRIEF DESCRIPTION OF DEFICIENCY (IF DEFICIENCY

BOX WAS CHECKED)

10. DETAILED PERFORMANCE ASSESSMENT

II. CONTRACTOR VALIDATION

11. CONTRACTOR REPRESENTATIVE CONCUR

NON-CONCUR

12. CORRECTIVE ACTION ESTIMATED COMPLETION

DATE

13. CONTRACTOR REPRESENTATIVE CORRECTIVE ACTION AND PREVENTION OF RECURRENCE OR REASON

FOR NON-CONCURRENCE OF COR CITED DEFICIENCY

III. ACTION CORRECTED

14. CONCUR NON-CONCUR COR SIGNATURE AND DATE

15. COR REMARKS (REQUIRED)

6. CONTRACTOR REPRESENTATIVE REMARKS

Appendix 3 – Customer Complaint Record

CUSTOMER COMPLAINT RECORD

DATE/TIME OF COMPLAINT

SOURCE OF COMPLAINT

ORGANIZATION

BUILDING NUM-

BER

INDIVIDUAL

PHONE NUMBER

NATURE OF COMPLAINT

CONTRACT REFERENCE

VALIDATION

DATE/TIME CONTRACTOR INFORMED OF COMPLAINT

ACTION TAKEN BY CONTRACTOR

RECEIVED/VALIDATED BY

Appendix 4 – Corrective Action Report

CORRECTIVE ACTION REPORT (CAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACTOR

2. CONTRACT NUMBER

3. TYPE OF SERVICES

4. FUNCTIONAL AREA

5. SUSPENSE DATE

6. CONTROL NUMBER

7. DEFICIENCY MAJOR MINOR

FINDING:

FINDING IMPACT:

Please respond with a written corrective action plan that details the corrective action of the cited deficiency, the cause of the defi-ciency, and actions taken to prevent recurrence by Suspense Date in Block 5. If date was not entered in Block 5, the contractor is not required to provide a response.

8. QUALITY ASSURANCE PERSONNEL (COR)

TYPED NAME AND GRADE

SIGNATURE AND DATE

9. ISSUING AUTHORITY

TYPED NAME AND GRADE

SIGNATURE AND DATE

10. COR RESPONSE TO CONTRACTOR CORRECTIVE ACTION AND ACTION TAKEN TO PREVENT RECURRENCE

11. COR DETERMINATION

ACCEPTED REJECTED

12. CLOSE DATE