A00001_ATTACHMENT 5 - COACA TECHNICAL PROPOSAL CONTENT REQUIREMENTS 70B06C20R00000096_4June2020.pdf

ATTACHMENT 5

70B06C20R00000096

Evaluation Factor

Proposal Tab

Information and materials to be provided by Offeror Page Number

Factor 1 Technical Approach

Tab 1:

Element 1:

Programming customizable

CAT

algorithms

Provide a detailed executive summary of the capability of the system for custom programming of Computer Adaptive Test (CAT) algorithms, the types of assessments and formats, test administration capabilities, and the data and other information captured by the system. Provide a listing and description of capability for the following:

Custom developed CAT algorithms including but not limited to:

1, 2, and 3‐parameter logistic models for dichotomous items

Differences in item pool sizes

User specified quadrature points and weights

Customizable Scaling factors

Customizable item selection methods

Customizable item exposure methods

Customizable CAT stopping rules

Content balancing options

Theta estimation approaches

Different initial theta values

Options for truncating theta estimates

Ability to limit theta estimate jumps by values or numbers of items

Provide samples/illustrative mock‐ups for:

At least 2 examples of previous customized CAT algorithms programmed

Example CAT algorithms should demonstrate ability to program adaptive algorithms for custom CATs

Reports or other documentation that describes the technical aspects of CAT engines that have been previously programmed for clients

Factor 1 Technical Approach

Tab 2:

Element 2:

System Capability

Provide a detailed executive summary of the system capabilities to include:

Random assignment of multiple series within a single applicant pool, Administering different tests within a test battery to applicants.

Capability of the system to:

Maintain item response latencies, Maintain supplemental materials, Allow applicants and administrators to access automated, communications, scheduling, and on‐demand access to test results, Types of administrator support capabilities available to include data access, data archiving, reports, and queries, Constrain accessibility by system roles and business rules.

The types of assessments and assessment formats supported including:

Cognitive and non‐cognitive assessments

Evaluation Factor

Proposal Tab

Information and materials to be provided by Offeror Page Number

Multiple choice, video‐based and interactive assessment content formats supported (e.g., gamification, draggable text/objects, drawing, matching).

Types of supplemental test materials supported (e.g., simulated emails, memos, photos, videos with sound, documents).

Proctor Manual

Applicant instructions, if available

Reports to be generated under this contract, the requirements for which are listed in the solicitation requirements (SOW).

Proctor Training Materials

Technical Approach

Tab 3 :

Element 3 Testing Site Network

Provide a detailed executive summary of the testing site network to include:

The number and location of test centers within the United States and internationally

The ability or plan to add new tests centers

The ability to set up mobile test sites where brick and mortar test sites do not exist

A summary of test centers that offer test sessions outside of normal business hours

Test site equipment standards (e.g., internet connectivity/bandwidth, hardware, and equipment standards, and explanations of how different sites may vary. The descriptions shall include the equipment available at static testing sites and any differences in equipment between sites.)

Configuration of static test centers; address how the set‐up and configuration requirements for test centers prevents cheating (e.g., space between testing stations, privacy screens, etc.)

Configuration of mobile test centers; address how the set‐up and configuration requirements for test centers prevents cheating (e.g., space between testing stations, privacy screens, etc.)

Testing center protocols (e.g., procedures for documenting administration protocol, unusual incident reporting procedures, emergency evacuations, power failures, computer failures, internet failures)

Testing center protocols (e.g., procedures for suspected and confirmed cheating and ADA accommodations)

Evaluation Factor

Proposal Tab

Information and materials to be provided by Offeror Page Number

Factor 1 Technical Approach

Tab 4:

Element 4:

Test Administration Methods

Provide a detailed executive summary of test administration methods to include:

Types of virtual proctoring offered (e.g., remote, forensic)

The technology on which unproctored assessments can be completed (e.g., mobile phone, PC)

Methods used to verify applicant identity

Methods used to ensure test security using remote proctored administration methods.

Remote test proctoring protocols (e.g., procedures for documenting administration protocol, unusual incident reporting procedures, power failures, computer failures, and internet failures, procedures for suspected and confirmed cheating and ADA accommodations)

Proctor Training Plan detailing how proctors will receive initial and yearly maintenance training on protocols for greeting applicants, positively identifying applicants, securing items prohibited from the test room, performing monitoring duties during testing, test administration, unusual circumstances, and privacy/information security

Technical Approach

Tab 5:

Element 5:

Web services and security, compliance, and privacy requirements.

Provide a detailed executive summary of the process used for web services and security, compliance, and privacy requirements, to include:

Demonstrating ability to transfer data to CBP and receive data from CBP that conforms to the following technical requirements:

Data must be made available via secure web services

Provide and authentication mechanism that ensures only authorized requests are processed

Web service should be available for use 24 hours a day, seven days a week

The ability to ensure that standardized testing environments meet industry standards for high‐volume, litigious (high‐stakes) testing environments across test sites.

Approach to safeguarding sensitive information

Demonstrate ability to establish a web services system to transfer data to CBP and from CBP.

Factor 2 Manage‐ ment Approach

Tab 6:

Element 1 Staffing/Key Personnel

Provide a detailed description of the personnel resources that will be employed for this project and demonstrate how these resources will meet solicitation requirements.

Demonstrate key personnel’s capability to meet the Government's requirements by providing position descriptions and qualifications for categories of personnel intended for this project or by inclusion of sample resumes.

Evaluation Factor

Proposal Tab

Information and materials to be provided by Offeror Page Number

Provide samples/illustrative mock‐ups for:

Resumes for all individuals identified as "key personnel" should be included and contain information on education, background, recent work experience, and specific scientific or technical accomplishments.

Resumes should be no longer than 2 pages each.

Factor 2 Manage‐ ment Approach

Tab 7:

Element 2:

Transition Plan

The Contractor shall provide a transition‐in and transition‐out plan by providing any required transition planning or program execution associated with the transition in and out timeline to include:

Willingness and agreement to transfer data, documents and records

Coordination with Government representatives

Review, evaluation and transition of current services

Transition of historic data

Transfer of all necessary business and/or technical documentation

Transfer of Government‐Furnished Information (GFI)

Provide samples/illustrative mock‐ups for:

Transition Plan documentation, should include a schedule, with appropriate milestone charts to identify timelines and critical path items involved.

Web Services Plan detailing how Contractor will establish a web services system for electronic data transfers for data provided to CBP and data provided by CBP

Factor 3 Past Perform‐ ance

Tab 8 Element 1 Quality of Product of Service

Offeror’s past performance on similar work to include programming, uploading, and delivering custom‐developed computer adaptive and static tests in high stakes testing environments. Offeror’s actual performance on all past performance submissions. In conducting the past performance assessment, the Government may use data obtained from other sources as well as that provided in the proposal. In the case of an Offeror without a record of past performance or for whom information on past performance is not available, the Offeror will not be evaluated favorably or unfavorably on past performance.

Quality of Product or Service: compliance with contract requirements, technical excellence and know‐how, successful services for programming computer adaptive testing algorithms, applicant scheduling, proctored and unproctored test administration including remote proctoring, and test scoring and reporting.

Tab 9 Element 2 Timeliness of Performance

Timeliness of Performance: meets milestones and deadlines, quick turnaround time on requests, promptly communicates issue, reliable, stays on schedule, responsive to technical direction, and completes tasks on time.

Tab 10 Element 3

Problem Resolution: anticipates, avoids, or mitigates problems;

satisfactorily overcomes or resolves problems; prompt notification of problems; and recommends viable solutions.

Evaluation Factor

Proposal Tab

Information and materials to be provided by Offeror Page Number

Problem Resolution

Factor 3 Past

Tab 12 Element 4 Business Relations

Business Relations: effective management; accountability for employee performance; business‐like concern for customer’s interests; effective use of web technology, effective supervision of staff; reasonable and cooperative behavior; flexibility; and maintains high employee morale.

Tab 13 Element 5 Customer Service

Customer Service: prompt responses and communication; understands and embraces service and program goals; team approach with the customer; satisfaction with the contractor’s services; positive customer feedback; courteous interactions; and initiative.