70B06C22R00000038 Attachment E- Quality Assurance Surveillance Plan.pdf

ATTACHMENT E

Quality Assurance Surveillance Plan (QASP)

RFP 70B06C22R00000038

Performance Requirements Summary

Performance standards define desired services. The Government performs surveillance to determine if the contractor exceeds, meets or does not meet these standards.

The following matrix includes performance standards. The Government will use these standards to determine contractor performance and will compare contractor performance to the Acceptable Quality Level (AQL).

The incentive for achieving the Acceptable Quality Levels (AQLs) listed in the Performance Requirements Summary below is a positive past performance evaluation, and it should be understood that failure to meet the performance metrics below will result in negative past performance evaluations. Past Performance Evaluations will be submitted to the Contractor Performance Assessment

Reporting System (CPARS) for all government agencies to review, and will contain detailed narratives explaining reasons for positive and negative assessments.

Performance

Based Task

Indicator Standard Acceptable Quality Level Method of Surveillance

System Security

PWS 1.5.4 a through f, h, i, and 1.5.4.1

System Security Segregated storage, third-party assessment (or SSP & RTM), Data encryption, Vulnerability scans, Security patches, destruction of CBLE data

No failure per six-month period of performance.

Contractor reporting in compliance with PWS.

Possible CBP-conducted Security Assessment and Section 7.1 Deliverables Table: Third-party assessment (or SSP & RTM), Monthly Status

Report, Monthly Vulnerability Scan, and

Certificate of Destruction

System Security

PWS 1.5.4 g

(also, see PWS sections 1.5.5.1, 1.5.5.2, 5.6.8 and

5.6.9)

Privacy, Rules of

Behavior, and

Security

Awareness

Training

All personnel with access to registrant PII complete training in accordance with PWS

No failure per six-month period of performance. Any failures must be resolved and approved by CBP.

Contractor reporting and inspection in compliance with PWS. Section 5.2 Milestone

Table and Section 7.1 Deliverables Table: DHS

Privacy, Rules of Behavior and Information

Technology Training completed.

Indicator Standard Acceptable Quality Level Method of Surveillance

Special

Qualifications

PWS 1.5.5

Criminal

Background

Investigation

All personnel on the contract must undergo a criminal background investigation

No failure per six-month period of performance

Contractor reporting. Section 7.1 Deliverables

Table: Monthly Status Report

Management

Plan PWS-5.1

Management

Plan Document

Describes planning actions and activities required to satisfy

PWS requirements

All activities and required items for the Management

Plan are clearly addressed in detail and contract is executed in accordance with the schedule outlined in the

Management Plan.

Contractor Reporting and Inspection. Section

5.2 Milestone Table and Section 7.1

Deliverables Table: Finalized Management Plan

Exam System

Requirements

PWS-5.3

Provide Exam

System

Exam system passes System

Acceptance Testing (SAT) and accommodates ADA requests

No failure per six-month period of performance. Any failures must be resolved and approved by CBP using the standard SAT practice.

100% Inspection; Customer feedback (zero complaints). Section 5.2 Milestone Table and

Section 7.1 Deliverables Table: Both in-person and remotely proctored Electronic CBLE

Delivery Systems

Exam

Administration

Requirements

PWS-5.4.1.1,

5.4.1.2, and

5.4.1.9

Exam

Registration and

Examinee communication

All registrants receive the 4 emails outlined in 5.4.1.9 and all registrants are assigned to a testing site

Not more than 1 failure per six-month period of performance.

Contractor reporting and customer feedback.

Section 5.2 Milestone Table and Section 7.1

Deliverables Table: Draft emails to examinees for CBP to approve; The Contractor provides draft ATPs for government review and approval for:

• The sample CBLE

• The site selection website (in-person)

• The exam appointment selection website

(remote)

• The electronic delivery CBLE system for the in-person exam administration

Indicator Standard Acceptable Quality Level Method of Surveillance

• The internet-based/remotely proctored

CBLE Exam System;

The Contractor contacts registrants via email with instructions on testing site selection (in person only), or with instructions on scheduling their remote exam; The Contractor contacts registrants via email reminding examinees to select a testing site, or to schedule their exam time; The Contractor finalizes registration changes; allows registrants to change their exam testing site and time selection if any spaces are open; The

Contractor issues updated admission tickets

(due to registration change); The Contractor contacts registrant via email with a reminder of examination, with check-in information, and behavior requirements.

Exam

Administration

Requirements

PWS-5.4.1.3,

5.4.1.4, 5.4.2.4, 5.4.3.4

ADA Requests and Examinee

Check-in

All approved ADA requests are accommodated, examinees are checked-in and are not turned away without approval from

CBP in accordance with the

PWS.

No failure per six-month period of performance.

Contractor reporting and customer feedback.

Section 5.2 Milestone Table and Section 7.1

Deliverables Table: ADA Reasonable

Accommodations Report; ADA Reasonable

Accommodations Notification

Exam

Administration

Requirements

PWS 5.4.1.6

War Room Notification to CBP via the War

Room of issues as they arise

No failure per six-month period of performance

Contractor reporting, customer feedback, inspection. Section 5.2 Milestone Table and

Section 7.1 Deliverables Table: The Contractor provides a virtual War Room

Indicator Standard Acceptable Quality Level Method of Surveillance

Exam

Administration

Requirements

PWS 5.4.1.6.1

Examinee

Cheating

Examinees caught cheating shall be allowed to continue the exam unless circumstances pose a threat to the integrity of the exam. Otherwise, proctor shall document the event and CBP will review at a later time. Prior to any termination of an exam, or turning away an examinee during check-in, the Contractor must contact CBP via the War

Room to receive prior approval.

No failure per six-month period of performance

Contractor reporting, customer feedback, inspection. Section 7.1 Deliverable Table: After

Exam Site Report

Exam

Administration

Requirements

PWS 5.4.1.6.2

(also see section

1.5.4 h)

Data Retention in case of Appeal or Court

Proceedings

If any data, including video, is retained for the in-person or remote proctoring exam administration and there is an appeal, including appeals related to a suspected event of cheating, then the contractor must maintain all data and video until as such time as directed by CBP to destroy it, or the Contractor must transfer all data in a common file format(s) to CBP via a CBP approved means.

No failure per six-month period of performance.

Contractor reporting, inspection

Indicator Standard Acceptable Quality Level Method of Surveillance

Exam

Administration

Requirements

PWS-5.4.2.1 –

5.4.2.3

Remote Exam

Registration website

Exam Registration Website passes System Acceptance

Testing (SAT) and examinees are registered for the exam in accordance with the PWS.

No failure per six-month period of performance. Any failures must be resolved and approved by CBP using the standard SAT practice.

Contractor reporting, customer feedback, inspection. Section 5.2 Milestone Table and

Section 7.1 Deliverables Table: The Contractor delivers site selection and exam time selection websites after completion of internal testing for

Government SAT and provides the link to examinees; The government provides the

Contractor with a list of registrants for the remotely proctored CBLE; Time-Selection List for the remotely proctored exam.

Exam

Administration

Requirements

PWS-5.4.2.5 and

5.4.3.6

Provide Exam

Proctors.

Remote: Maximum Proctor ratio 1:15.

In-Person: Maximum Proctor ratio 1:25.

If Contractor is granted approval to hire non-U.S.

Citizens, then the Contractor shall abide by requirements to hire legal permanent residents.

No failure per six-month period of performance. Any failures must be resolved and approved by CBP using the standard SAT practice.

Contractor reporting, sampling observations at test site locations, and inspection of quality control records.

Exam

Administration

Requirements

PWS-5.4.2.5 and

5.4.3.6

Proctor

Citizenship

If Contractor is granted approval to hire non-U.S.

Citizens, then the Contractor shall abide by requirements to hire legal permanent residents as outlined in PWS.

No failure per six-month period of performance.

Contractor reporting: Maintain an up-to-date file of all non-U.S. Citizens’ information outlined in PWS section 5.4.2.5 and provide a copy to the government upon request.

Indicator Standard Acceptable Quality Level Method of Surveillance

Exam

Administration

Requirements

PWS-5.4.2.6,

5.4.2.7, and

5.4.3.8

Proctor behavior Proctor shall be quiet (no background noise) and prevent cheating in accordance with the

PWS

No failure per six-month period of performance.

Contractor reporting, customer feedback, inspection

Exam

Administration

Requirements

PWS-5.4.2.7 and

5.4.3.10

A quiet, self-contained room

A quiet, self-contained room is defined as a room where no noise or distractions considered unsatisfactory for a testing environment could be heard and is a room with four (4) permanent structured walls (no partitions). Only dedicated testing facilities or computer labs, not hotel conference rooms or make-shift rooms created with a partition.

No failure per six-month period of performance.

Customer feedback

Exam

Administration

Requirements

PWS-5.4.2.8 –

5.4.2.10, and

5.4.2.13

Examinee disconnection procedures and

Connectivity compatibility testing

All electronic capabilities to ensure the examinee can take the test remotely

No failure for 5.4.2.8-

5.4.2.10 per six-month period of performance. For

5.4.2.13: Works with examinee through at least 3 disconnects or a single disconnection of 10 minutes per examinee.

Contractor reporting, customer feedback, inspection

Indicator Standard Acceptable Quality Level Method of Surveillance

Exam

Administration

Requirements

PWS-5.4.3.1-

5.4.3.3, 5.4.2.11 and 5.4.2.12

In-Person Exam

Site Selection

Scheduling and

Registration;

Remote Exam

Scheduling

Site selection website passes

System Acceptance Testing

(SAT). Examinee are registered for the exam in accordance with the PWS. List of scheduling windows for

Remote exam

No failure per six-month period of performance

Contractor reporting, customer feedback, inspection. Section 7.1 Deliverables Table: A draft list of testing time zone windows, including the times within the scheduling windows for the remotely proctored exam;

Final List of Testing Site Locations and Facility

Descriptions. A final list of testing time zone windows, including the times within the scheduling windows;

Site-Selection for the in-person exam and Time-

Selection for the remotely proctored exam are completed;

Testing Site-Selection List for the in-person exam. Time-Selection List for the remotely proctored exam;

The Contractor delivers site selection and exam time selection websites after completion of internal testing for Government SAT and provides the link to examinees.

Exam Site

Requirements

PWS-5.5.1, and

5.5.3 - 5.5.5

Provide Exam

Sites, equipment, and materials.

All Exam Sites, Equipment, and

Materials (including exam delivery system) available as required in the PWS.

Not more than 1 failure per six-month period of performance.

Contractor reporting, Sampling at test site locations and customer feedback, inspection of quality control records. Section 7.1

Deliverables Table: Draft List of Testing Site

Locations and Facility Descriptions for the in-person exam; Final List of Testing Site

Locations and Facility Descriptions. A final list of testing time zone windows, including the times within the scheduling windows.

Indicator Standard Acceptable Quality Level Method of Surveillance

Exam Site

Requirements

PWS-5.5.2, and

5.3.1.a

Exam Site internet connection

Exam Sites meet minimum download and upload speeds, without noticeable loading or time lag in displaying information after input from the system user

Not more than 1% of the total in-person examinee population reports an internet connectivity failure per six-month period of performance.

Customer Complaints, Contractor reporting, and inspection

Program

Management

Requirements

PWS-5.2, 5.6.1 -

5.6.4, and 7.1

Program

Management

All activities are performed in accordance with the PWS and the approved Program

Management Plan, including all milestones tasks and deliverables completed according to established timelines.

Not more than 1 failure per six-month period of performance.

Contractor reporting, observation, customer feedback, inspection of quality control records.

Section 5.2 Milestone Table and Section 7.1

Deliverables Table: Kick-off Meeting and

Meeting Minutes; Weekly Coordination

Meeting Agenda; Weekly Coordination

Meeting; Weekly Coordination Meeting Notes;

Monthly Status Reports

Program

Management

Requirements

PWS-5.6.5 and

5.6.6

NDAs All Contractor staff with access to or that will see examinee PII or SPII shall sign a DHS NDA in accordance with PWS

No failure per six-month period of performance

Contractor reporting and inspection. Section 5.2

Milestone Table and Section 7.1 Deliverables

Table: Provide NDAs.

Program

Management

Requirements

PWS-5.6.7

Sample Exam

Website

URL link provided and passes

SAT in accordance with PWS

No failure per six-month period of performance

Contractor reporting and inspection. Section 7.1

Deliverables Table: Provide Sample CBLE for public exam practice.

Indicator Standard Acceptable Quality Level Method of Surveillance

Program

Management

Requirements

PWS-5.6.10 and

5.6.11

Communication All issues communicated in a timely and transparent manner

No failure per six-month period of performance.

Inspection and Observation. Section 5.2

Milestone Table.

Program

Management

Requirements

PWS-5.6.12 -

5.6.14

Reporting All reports delivered in an accurate and timely manner

No failure per six-month period of performance.

Inspection. Section 5.2 Milestone Table and

Section 7.1 Deliverables Table: After Exam

Site Report and Exam No-Show Report; Initial

Item Analysis Report; Final Exam Item

Analysis; Site Score Report, Survey Data

Report, Raw Data Report, Individual Answer

Records.