HSTS01-15-R-HRM015.pdf

SOLICITATION/CONTRACT/ORDER FOR COMMERCIAL

ITEMS

QUOTER TO COMPLETE BLOCKS 12, 17, 23, 24, & 30

1. REQUISITION NUMBER

21-15-205HRM015

2. CONTRACT NO.

3.AWARD.EFFECTIVE

DATE

4. ORDER NUMBER

5. SOLICITATION NUMBER

HSTS01-15-R-HRM015

6. SOLICITATION

ISSUE DATE

6/19/2015

7. FOR SOLICITATION

INFORMATION

CALL:

a. NAME

Victor Carden Victor.Carden@tsa.dhs.gov

b. TELEPHONE NUMBER

(571) 227-4834

8. QUOTE DUE DATE/

LOCAL TIME

07/20/2015

4:00 pm EST

9. ISSUED BY CODE

10. THIS ACQUISITION IS 11. DELIVERY FOR

FOB

12. DISCOUNT

TERMS

U.S. DEPARTMENT OF HOMELAND SECURITY

TRANSPORTATION SECURITY

ADMINISTRATION

601 SOUTH 12

th

STREET

ARLINGTON, VA 20598-6025

UNRESTRICTED DESTINATION UNLESS

BLOCK IS MARKED

SET ASIDE: 100%FOR SEE SCHEDULE Net 30 Days

SMALL BUSINESS

SMALL DISADV. BUSINESS

13a. THIS CONTRACT IS A RATED

ORDER UNDER DPAS (15 CFR 700)

8(A) 13b. RATING

NAICS/SIC: 541612

SIZE STANDARD: $15 Million

14. METHOD OF SOLICITATION

RFQ IFB RFP

15. DELIVER TO See Statement Of Work CODE 16. ADMINISTERED BY CODE 01

Human Capital &Finance 17a.

CONTRACTOR/QUOT

ER

CODE FACILI

TY

18a. PAYMENT WILL BE MADE BY

CODE 51800

U.S. Coast Guard Finance Center

TSA Commercial Invoices

P.O. Box 4111

Chesapeake, VA 23326-4111

Email: FIN-SMB-TSAInvoices@uscg.mil Fax:757-413-7314

17b. CHECK IF REMITTANCE IS DIFFERENT AND PUT SUCH

ADDRESS IN QUOTE

18b. SUBMIT INVOICES TO ADDRESS SHOWN IN BLOCK 18a UNLESS BLOCK

BELOW IS CHECKED. SEE ADDENDUM

19.

ITEM NO.

20.

SCHEDULE OF SUPPLIES/SERVICES

21.

QUANTITY

22.

UNI

T

23.

UNIT PRICE

24.

AMOUNT Web-based Assessment Testing

25. ACCOUNTING AND APPROPRIATION DATA

26. TOTAL AWARD AMOUNT (For Govt.

Use Only)

27a. SOLICITATION INCORPORATES BY REFERENCE FAR 52.212-1, 52.212-4. FAR 52.212-3 AND 52.212-5 ARE ATTACHED. ADDENDA ARE ARE NOT

27b. CONTRACT/PURCHASE ORDER INCORPORATES BY FULL TEXT FAR 52.212-4 & FAR 52.212-5. ADDENDA ARE ARE NOT ATTACHED.

28. CONTRACTOR IS REQUIRED TO SIGN THIS DOCUMENT AND RETURN 1 COPIES TO

ISSUING OFFICE. CONTRACTOR AGREES TO FURNISH AND DELIVER ALL ITEMS SET

FORTH OR OTHERWISE IDENTIFIED ABOVE AND ON ANY ADDITIONAL SHEETS

SUBJECT TO THE TERMS AND CONDITIONS SPECIFIED HEREIN.

29. AWARD OF CONTRACT: REFERENCE QUOTE

DATED . YOUR QUOTE ON SOLICITATION (BLOCK 5).

INCLUDING ANY ADDITIONS OR CHANGES WHICH ARE SET FORTH

HEREIN, IS ACCEPTED AS TO ITEMS:

30a. SIGNATURE OF QUOTER/CONTRACTOR

31a. UNITED STATES OF AMERICA (SIGNATURE OF CONTRACTING OFFICER)

30b. NAME AND TITLE OF SIGNER (TYPE OR

PRINT)

30c. DATED

SIGNED

31b NAME OF CONTRACTING OFFICER (TYPE OR PRINT)

Shawnna L. Sims 31c. DATE

SIGNED

32a. QUANTITY IN COLUMN 21 HAS BEEN 33. SHIP NUMBER

34. VOUCHER NUMBER 35.AMOUNT

VERIFIED

CORRECT

FOR

RECEIVED INSPECTED ACCEPTED, AND CONFORMS

TO THE CONTRACT, EXCEPT AS NOTED

PARTIAL FINAL

36. PAYMENT

COMPLETE PARTIAL FINAL

37. CHECK

NUMBER

32b.SIGNATURE OF AUTHORIZED GOVT.

REPRESENTATIVE

32c. DATE 38. S/R ACCOUNT NUMBER

39.S/R VOUCHER

NUMBER

40. PAID BY

42.a. RECEIVED BY (Print)

41a. I CERTIFY THIS ACCOUNT IS CORRECT AND PROPER FOR PAYMENT

41b. SIGNATURE AND TITLE OF CERTIFYING

OFFICER

41c. DATE

42b. RECEIVED AT (Location)

42.c DATE REC’D

(YY/MM/DD)

42d. TOTAL CONTAINERS mailto:Victor.Carden@tsa.dhs.gov mailto:FIN-SMB-TSAInvoices@uscg.mil

HSTS01-15-R-HRM015 Page 2

SECTION I - SUPPLIES OR SERVICES AND PRICES/COSTS

1. DESCRIPTION OF SERVICES

The Contractor shall provide all management, supervision, manpower, supplies, equipment

(excepted as specified herein), and shall have the capability to meet all requirements specified in the Statement of Work.

2. TYPE OF CONTRACT

This is an Indefinite-Delivery/Indefinite-Quantity (IDIQ) contract, as identified under Federal

Acquisition Regulation (FAR) 16.504. The Government anticipates that Task Orders will be issued on a Firm-Fixed Price (FFP) basis and may include a Cost CLIN(s) to support Other

Direct Costs (ODC) (e.g. non-labor requirements such as travel). No profit shall be applied by the Contractor in support of ODCs. Time and Materials/Labor Hour (T&M/LH) Task Orders may be issued if the Government determines that a FFP Task Order is not suitable.

1) The guaranteed minimum to be ordered under this contract will be met under Task Order 001

(provided as Attachment 3) which covers phase-in activities, certification and accreditation, and monthly program services to support those activities. During the period of performance of this

Contract, the Government may issue one or more Task Orders. The Government’s intent is to issue Task Order 001 at the same time as the IDIQ contract. The Government is not obligated to order all CLIN items.

2) The total maximum quantity of services to be awarded during the life-cycle of this contract is estimated to be valued at $70 million.

When responding to an actual Task Order Request (TOR) under this IDIQ contract, the

Contractor shall use the fixed price labor rates/categories, Fixed Unit Prices, and catalog price list(s) contained in this contract at Attachment 1.

3. PERIOD OF PERFORMANCE (PoP)

The ordering period of this indefinite delivery indefinite quantity (IDIQ) contract is five (5) years. Task Orders may be issued at any time during the five (5) year ordering period at the fixed unit prices established for the corresponding contract year. In addition, the contract includes the potential awarding of three twelve-month Award Terms for a total possible eight year PoP. TSA seeks to provide additional incentives for the contractor to perform at a level

TSA considers better than acceptable. The award term will only be exercised if overall performance is evaluated at a level greater than satisfactory in meeting contractual requirements

(See Attachment 9 Award Term Plan).

4. TASK ORDER FUNDING

Funds for services provided will be obligated at the task order level.

5. CONTRACT PRICING: This pricing section includes all pricing arrangements that the contractor shall use to price Task Order Requests. The Contractor shall use the labor rates, firm-fixed prices, and fixed unit prices established herein for all Task Order Requests. It is not possible to determine the actual number of assessments to be procured, the number of suitable

HSTS01-15-R-HRM015 Page 3 tests/assessments to be included in the Catalog, or the need for integration with other human resource systems during the contract period. The Government cannot specify any definite quantity of services. The actual quantity/value of and time frame when task orders will be placed will vary. Contract Pricing is for the purpose of pricing task orders as well as the initial evaluation and award of the contract.

A. LABOR CATEGORIES AND LABOR RATES

The labor categories and labor rates provided at Attachment 1 “CATS Contract Pricing” shall be used for the negotiation of Task Orders for services not otherwise priced out in CLIN price schedule for the development of new assessments and new items. The labor categories and fixed-price labor rates are inclusive of all cost factors (e.g., direct labor, indirect labor, G&A, and profit).

B. CATALOG PRICE LIST (ASSESSMENTS) The catalog of fixed-unit prices

(or fixed delivery fees) per assessments are provided at Attachment 2 “Catalog of Assessments”.

New assessments may be added to the CATS during contract performance when approved by the

Government.

C. CONTRACT LINE ITEM NUMBER (CLIN) PRICING

A CLIN pricing schedule is provided at Attachment 1 “CATS Contract Pricing”. All prices are inclusive of personnel, equipment, facilities, and services to meet the full scope of this requirement on a fixed unit price basis.

HSTS01-15-R-HRM015 Page 4

SECTION II - STATEMENT OF WORK (SOW)

1 GENERAL

1.1 Background

The mission of the Transportation Security Administration (TSA) is to protect the United States

(U.S.) transportation systems and to ensure freedom of movement for people and commerce.

TSA sets the standard for excellence in transportation security through people, processes, and technology, and is committed to promoting a culture founded on the values of integrity, innovation, and team spirit. In conjunction with state, local, and regional partners, TSA oversees security for the nation’s highways, railroads, buses, mass transit systems, seaports, and 450+ airports.

The TSA, Office of Human Capital (OHC), Organizational Assessment Services (OAS) Branch, is responsible for developing, implementing and analyzing assessments for selection, promotion, and certification for occupations within the agency in order to ensure that TSA candidates and employees possess the knowledge, skills, abilities, and competencies necessary to carry out assigned duties.

On November 19, 2001, the Aviation and Transportation Security Act (ATSA) established the existence of TSA, as well as performance criteria for Transportation Security Screeners, now known as Transportation Security Officers (TSO). In accordance with ATSA requirements, TSOs are held accountable for their technical knowledge, as well as their ability to use that knowledge to perform the essential functions of the Security Officer Job. The Employee

Assessment Management System (EAMS) is the tool TSA currently uses to assess job knowledge. These assessments are comprised of questions drawn from TSA’s Standard

Operating Procedures (SOP) guidance, training curriculum, and other relevant materials. These tests vary depending on the specific function performed by a TSO and are typically scored as pass/fail.

Additionally, position announcements throughout the year are responded to by hundreds of thousands of applications, many of which must be screened and may require further evaluation to facilitate selection of the most highly qualified individuals. The OAS Branch is responsible for developing and implementing tests and assessments to evaluate candidates.

Previously, TSA has not had a unified system that can be used to develop, administer and analyze assessments for selection, promotion, and certification. To this end, this acquisition seeks to establish a Catalog of Assessments and Testing System (CATS) to ensure that assessments can be administered in a timely manner to applicants and individuals across the agency, while maintaining or improving the efficiency and effectiveness of these processes. The acquired testing system must offer a library of assessments that can be readily accessed and administered to applicants through the CATS system, and must have the capability to administer

TSA and third-party custom-developed assessments. The testing system must also have the

HSTS01-15-R-HRM015 Page 5 capacity to integrate results from any or all of these assessments with our hiring (Government furnished and third-party) and Learning Management System (LMS) and offer support for TSA customers.

The following historical information is provided for reference only:

Required Tasks

Calendar Year (CY)

2012 Volume per

Annum

CY 2013 Volume per Annum

CY 2014 Volume per Annum

Assessment Development 1 2 0

Computer Based Testing

(CBT) – Pre-Hire

Assessments

71,572 34,478 55,955

CBT Help Desk – Inbound

Call 46,057 24,535 37,106

CBT Help Desk – Email 5,014 2,179 2,627

CBT Help Desk – Email w/Attachment 58 25 36

CBT Help Desk – Fax 4 0 1

CBT Help Desk – Mail 3 0 1

CBT Help Desk – Other 14 910 175

CBT Help Desk – Outbound

Call 21,018 3,629 3,241

CBT Help Desk – Outbound

Mail 56 14 8

CBT Help Desk – Data

Import 0 0 1

Help Desk (Post-Hire

Assessments) Unavailable 1,437

1,632 (Jan – Oct

2014)

Offline Solution – Scanning

Volume (Post-Hire 2,277 1,222 287 (Jan-Oct 2014)

HSTS01-15-R-HRM015 Page 6

Assessments)

Post-Hire Assessments

(certification testing, etc.)

135,270 175,752

130,508 (Jan – Oct

2014)

1.2 Scope of Work

This Statement of Work (SOW) establishes the requirements for CATS for the TSA, OHC to create, deliver, administer, and manage assessments for TSA’s workforce positions, as required.

The CATS system will be used by TSA to evaluate candidates and incumbents within a variety of positions including TSOs; Screening Partnership Program (SPP) screeners; Federal Air

Marshals (FAMs); management, administrative, and professional (MAP) positions; and executive positions. These positions will be in a variety of business, technical, and security operations-related disciplines at the entry-level, team lead, first-line supervisor, manager, and executive levels.

The CATS shall be Commercial-off-the-Shelf (COTS) technology which must seamlessly integrate and interface with all TSA hiring systems (Government furnished and third-party) and the LMS. The CATS shall be capable of utilizing test content from TSA-developed assessments, the CATS provider’s library of assessments, and off-the-shelf and custom developed assessments from third-party sources. The system will have the ability to create and edit assessments and shall have sufficient bandwidth allowing for system response time that does not negatively impact the user experience and scalable storage capability to meet TSA’s testing storage needs.

The system will have the capability for OHC to author and revise tests and test items, administer tests, analyze test results, and generate reports. The system will also be used to assist with delivering initial certification and recertification assessments for TSA positions. The system will be capable of administering tests in an ‘offline’ or alternative environment (e.g., paper and pencil format, using scannable answer sheets, etc.) that is accessible by all sites as needed.

Administration methods for assessments shall include testing in a physically proctored setting – either at a Contractor-proctored and managed test site or at a TSA site with TSA personnel, un-proctored internet testing, and, in limited circumstances, offline or alternative environment (e.g., paper and pencil) testing at a proctored site. The Contractor shall provide proctors (i.e., test administrators) who will administer applicable tests in accordance with TSA standards.

Certification assessments will be mainly TSA developed and proctored by TSA. However, assessments for TSA candidates and employees can be TSA developed, provided through the

Contractor’s catalog, Contractor developed, or developed by a third-party. Assessments for candidates will generally be administered at a Contractor-proctored test facility. Assessments for current TSA employees shall be administered either at a Contractor proctored test facility or at a

TSA proctored location.

HSTS01-15-R-HRM015 Page 7

The Contractor shall develop, conduct and provide proctor/test administrator training for

Contractors and TSA personnel. The Contractor shall provide Help-Desk support for TSA customers.

The CATS shall have a self-service scheduling system that will allow TSA approved test-takers to self-schedule, cancel, and reschedule testing sessions at a Contractor-managed or TSA managed test site using secure, individual, online accounts. This self-scheduling system shall have the capability to send system generated email notifications to test-takers. TSA-managed test sites may not be available in all cases.

Contractor-managed test sites shall be situated at various regional locations and major metropolitan areas across the fifty (50) U.S., the District of Columbia (DC), and all U.S.

territories and possessions. Testing facilities at static sites should be within 60 miles of each airport, FAMs Field Office, and other TSA facilities, or provide mobile sites when static site positioning is not possible. Testing sites must have sufficient seating/testing capacity commensurate with TSA requirements for a specific airport/field office. Any internationally based proctored assessments shall be managed on a case-by-case basis. Contractor managed test sites must be able to accommodate undercover Federal law enforcement officers required to carry concealed weapons into the testing center while on duty.

TSA expects 125,000 certification tests annually for TSA’s current TSO workforce. TSA anticipates that roughly 100,000 of these assessments will be conducted online and 25,000 offline each year. Additionally, for selection and promotion assessments, TSA anticipates

58,000 annually for TSO candidates, 15,000 annually for MAP applicants 9,000 FAMs annually, and 500 executive applicants annually. All of these numbers may vary from year to year. The

CATS system will support the proper safeguarding and handling of Sensitive Security

Information (SSI), Sensitive Personally Identifiable Information (SPII) and all other sensitive information, per TSA guidelines.

1.3 Objective

The objective of this requirement is for the TSA to acquire a qualified Contractor to provide: a proven automated testing platform to test (online and offline), evaluate, and track assessments;

access to a library of commercially available assessments that are professionally developed and validated; Contractor expertise to identify and recommend assessments as necessary; custom developed and validated assessments; Contractor-managed tests sites; a self-service scheduling system that can be loaded with TSA and/or Contractor-established test sites and available times;

the ability to load TSA’s assessments onto the platform; and the ability to load and/or integrate with third-party assessments. It is the goal of this contract to establish a unified platform capable of managing all TSA assessments that can be integrated with TSA’s Hiring (Government furnished and third-party) and LMS. The integration with the LMS and upload of TSA-owned test content shall be prioritized to support TSA certification testing.

HSTS01-15-R-HRM015 Page 8

2 SPECIFIC REQUIREMENTS/TASKS

2.1 General System Requirements

Any licenses procured for the CATS and during the life of the Contract shall be procured on behalf of TSA, assigned to TSA and transferred to TSA. The Contractor shall procure all licenses for government use purposes to the maximum extent practicable.

2.1.1 The system shall:

2.1.1.1 Along with all TSA data, be stored on a physically segregated server from all other entities, both federal and non-federal and meets the requirements of the Certification and Accreditation (C&A). An exception to this subparagraph will be considered on a case-by-case basis (e.g. service provider has a testing system TSA already uses currently in place).

2.1.1.2 Have COTS technology.

2.1.1.3 Have the capability to seamlessly and securely integrate/interface and exchange structured data via web services with TSA or other Contractor hosted Government systems, to include TSA’s Hiring (Government furnished and third-party) and LMS, including:

2.1.1.3.1 Provide a scheduled daily automated transmission of structured

CATS data to the Integrated Hiring Solution (IHS).

2.1.1.3.2 Provide a scheduled daily automated transmission of structured

CATS data to the LMS.

2.1.1.3.3 Provide a scheduled daily automated transmission of structured

CATS data to the Recruiting and Hiring System (R&H).

2.1.1.3.4 Maintain security and encryption standards per Section 2.1.2 and

Section 8 of this SOW while data is “at rest”, or “in motion”.

2.1.1.3.5 With the Government’s prior approval - develop/implement and maintain additional manual/automated data transmissions and/or data exchanges in response to enterprise architecture changes, system usability issues, or physical/technology security considerations during the lifecycle of the project.

2.1.2 The Contractor shall:

2.1.2.1 Transform and load legacy assessment data provided by TSA into the

HSTS01-15-R-HRM015 Page 9

CATS database. The Contractor shall load all candidate and employee data from the previous two full calendar years and shall load data for all

TSO CBT results for failed assessments from 2009 to current.

2.1.2.2 Develop a Scale Test Plan, and validate that the legacy data has been loaded correctly into the CATS database.

2.1.2.3 Maintain the system, provide technical support, and resolve technical issues. The Contractor shall not exceed four late responses per calendar year and the Contractor shall not exceed two business days to respond to either phone call or email from TSA.

2.1.2.4 Be required to maintain data controls and protections as required by the

Privacy Act of 1974, as amended, and title 49, Code of Federal

Regulations (CFR) part 1520, as well as all applicable Department of

Homeland Security (DHS) and TSA information security requirements.

Further information can be found at http://www.dhs.gov/dhs-security-and-training-requirements-contractors

2.1.2.5 Encrypt all sensitive data transmitted outside system boundaries using

Federal Information Processing Standard (FIPS) 140-2 validated cryptographic modules, and algorithms compliant with FIPS 197

(Advanced Encryption Standard (AES) 256).

2.2 Test Item and Structure

2.2.1 The system shall have the functionality to allow TSA to configure each test/assessment using the following as needed and to the extent that commercially published tests allow:

2.2.1.1 Allow a variety of test item formats (e.g., multiple-choice, multiple-response, matching, ranking, audio and video, etc.).

2.2.1.2 Present test items as one item per screen and multiple items per screen.

2.2.1.3 Present the item stem as well as the item response options on one screen with the capability to display the item stem on multiple pages with the associated items for appropriate item types.

2.2.1.4 Enable, or disable, a display of the test time remaining and show a progress bar.

2.2.1.5 Randomize the order of test items.

2.2.1.6 Offer computer-adaptive testing.

http://www.dhs.gov/dhs-security-and-training-requirements-contractors http://www.dhs.gov/dhs-security-and-training-requirements-contractors

HSTS01-15-R-HRM015 Page 10

2.2.1.7 Present instruction screens to allow for learning system navigation after the candidate logs onto the system and provide one or two sample test items for each of the item types contained in the test. For TSA developed tests, TSA will provide sample items; for third-party or COTS test content, TSA will use published test instructions.

2.2.1.8 At the beginning of each assessment or test battery, require test-takers to acknowledge a confidentiality agreement stating that they will not share information about the assessment or test items.

2.2.1.9 Display question number (i.e., the number of the question to which the test taker is currently responding) and total number of questions in the assessment (e.g., question 33 of 50).

2.2.1.10 Permit or restrict the ability to allow test-takers to navigate back to previous questions.

2.2.1.11 Permit or restrict the ability to allow test-takers to skip test questions.

2.2.1.12 Permit or restrict the ability to allow test-takers to mark questions for later review.

2.2.1.13 Permit or restrict the ability to allow test-takers to search for items missing a response.

2.2.1.14 Require a response to all questions before test taker can exit the assessment.

2.2.1.15 Provide automatic enforcement of per assessment time limits and total assessment session time limits, and allow these time limits to be extended by an administrator to provide a reasonable accommodation on a per test taker basis when approved. Total assessment session duration varies depending on assessments selected for inclusion.

2.2.1.16 Allow supplemental test materials (e.g., simulated emails, memos, documents) to be presented to the candidate.

2.2.1.17 Allow individual tests to be presented as a test battery with assessments being provided in a fixed order within each battery.

2.2.1.18 Randomly assign test items to test takers based on content domain (e.g.

random number form generation, random question generation, serial form generation).

HSTS01-15-R-HRM015 Page 11

2.2.1.19 Present test takers with an exit survey, to be approved or provided by

TSA, upon conclusion of each assessment or test battery.

2.2.2 Authoring Requirements

2.2.2.1 The Contractor shall provide a system with the capability that allows TSA to create, edit, upload and approve multiple assessments. CATS shall be capable of providing the following:

2.2.2.1.1 Contractor support for uploading and formatting multiple assessments provided by TSA in a mutually agreed upon format.

2.2.2.1.2 Customizable multi-author environment.

2.2.2.1.3 Tracking authors.

2.2.2.1.4 Tracking edits (e.g., version control, author).

2.2.2.1.5 Randomization of assessment questions and answer options within assessment.

2.2.2.1.6 Moving/Re-Sequencing test questions (manually and/or automated).

2.2.2.1.7 Allowing designated TSA role-based users to author instructions that test takers review before, during, and after a test.

2.2.2.1.8 Allowing designated TSA role-based users to determine what feedback test takers see upon test completion, to include but not be limited to pass/fail, raw score, percent correct.

2.2.2.1.9 Using the standard copy/paste functions to enter test content.

2.2.2.1.10 Linking assessment items to topic areas/learning objectives.

2.2.3 Item Banking

2.2.3.1 The Contractor shall provide a system with item banking capability that allows TSA to securely store and retrieve TSA-provided test items.

CATS shall provide the following capabilities for designated TSA role-based users and Contractor support to:

2.2.3.1.1 Upload newly created test items to the item bank.

HSTS01-15-R-HRM015 Page 12

2.2.3.1.2 Maintain historical items in the item bank.

2.2.3.1.3 Provide the ability to copy or up-version items.

2.2.3.1.4 Ensure that items cannot be automatically written over once the test is finalized.

2.2.3.1.5 Allow users to manually select items from the item bank to be used in test generation based on the psychometric characteristics of items (e.g., item difficulty, item discrimination, item total correlation, number of times item has been included in a test, number of response options for the item, date item was created, and author).

2.2.3.1.6 Randomly assign test items from an item bank based on the psychometric characteristics of items (as specified in 2.2.3.1.5) to single or multiple assessments.

2.2.3.1.7 Display a list of all of the tests that have used that item for each item in the item bank.

2.2.3.1.8 Provide the ability to run aggregated item-statistic reports for each individual item across all tests.

2.3 Administrative Requirements

2.3.1 Scheduling. The system shall:

2.3.1.1 Have web-based automated registration, scheduling, and score reporting service, interface and transfer data bi-directionally with all TSA Hiring

(Government furnished and third-party) Systems and the LMS.

2.3.1.2 Update list and assign test batteries to test-takers as updates are made in

TSA’s hiring systems and LMS.

2.3.1.3 Permit TSA test-takers to self-register and schedule testing session(s), and when needed schedule the session at a Contractor-managed test site, or a

TSA-managed test site.

2.3.1.4 Allow test-takers the capability to schedule during TSA defined scheduling periods and cancel/reschedule test sessions (in accordance with TSA business rules that will be established or modified post contract award. Business rules will govern number of cancellations allowed, late arrivals, lifetime failures, etc.).

HSTS01-15-R-HRM015 Page 13

2.3.1.5 Allow test-takers to view test session appointment information, including date, time, and address.

2.3.1.6 Allow simultaneous file viewing by multiple authorized users.

2.3.1.7 Allow multiple test-takers to concurrently register for and schedule assessments.

2.3.1.8 Automatically confirm the candidate(s) scheduled test session, with TSA approved email and/or text message content via a non-responder or unattended email/text address, within one (1) hour of scheduling an assessment. The confirmation email/text shall include the test date and time, location, and test site requirements (e.g. the IDs needed for identification, length of test, etc.). The reminder email/text shall also include instructions for canceling or rescheduling the test session.

2.3.1.9 Allow TSA to make revisions to the notification messages to test takers as needed. Requested changes shall be completed within five (5) business days of the request.

2.3.1.10 Have the capability for the system to notify test-takers of changes to scheduled test sessions.

2.3.1.11 Send a TSA approved reminder email/text to the candidate within 48 – 72 hours prior to the scheduled test session. The reminder email/text shall include the test date and time, test site location, and test site requirements as described above. The reminder email/text shall also include instructions for canceling or rescheduling the test session.

2.3.1.12 Allow test-takers with disabilities the option to request reasonable accommodation (in accordance with the Americans with Disabilities Act

(ADA)) prior to or at the time of scheduling a test. The system shall also allow test-takers to cancel a request for an accommodation. The system shall automate notification to TSA and provide TSA with the ability to approve or disapprove an accommodation.

2.3.2 Test Administration. The Contractor shall:

2.3.2.1 Administer computer-based, online assessments in physically proctored, Contractor-owned/managed facilities within 60 miles of each airport, Federal Air Marshal Service (FAMS) Field Office, and any other TSA facility located in any of the 50 U.S., the DC, and any and all U.S.

territories and possessions.

HSTS01-15-R-HRM015 Page 14

2.3.2.2 Provide mobile testing sites when Contractor-owned/managed physically proctored static test sites are not available or will not meet TSA’s needs.

2.3.2.3 Ensure that all Contractor-owned/managed test sites, whether static or mobile, have seating/testing capacity commensurate with TSA requirements for specific airports, field offices, and other TSA facilities.

2.3.2.4 Provide TSA with the capability to administer computer-based online assessments and, if needed, offline or alternative administration methods, in physically proctored environments located on TSA properties, or on properties leased or rented by TSA for purposes of administering assessments to test takers.

2.3.2.5 Upload data accumulated through offline assessment administration into the CATS system.

2.3.2.6 Convert an online assessment to a format that can be administered via paper-and-pencil test administration, or another test administration format that is accessible by all sites as needed.

2.3.2.7 For tests and assessments to be administered at a TSA property by TSA personnel, the Contractor shall provide authorized TSA personnel with access to TSA approved materials required to administer the test on an as needed basis (no more than one late delivery and no more than two business days late per calendar year), such as:

2.3.2.7.1 Test administration instructions.

2.3.2.7.2 Answer keys (if applicable).

2.3.2.7.3 Scoring protocols (if applicable).

2.3.2.7.4 Instructions to test-takers.

2.3.2.7.5 Background materials, role-play scenarios and related test content

(if applicable).

2.3.2.7.6 Evaluation guidelines and any other materials that are required to administer and score the assessments.

2.3.2.8 Provide a secure testing environment that restricts test takers’ access to the following while in the test environment, as applicable:

2.3.2.8.1 Other tests.

HSTS01-15-R-HRM015 Page 15

2.3.2.8.2 Other applications.

2.3.2.8.3 Print functions.

2.3.2.8.4 Copy and paste functions.

2.3.2.8.5 Programs.

2.3.2.8.6 Other Internet sites.

2.3.2.9 At each Contractor-owned/managed test administration facility, provide a proctor to monitor the test and follow documented testing protocols. Test proctors at contracted locations shall wear name badges or shall otherwise identify themselves as Contractors in support of TSA in any in-person interaction or correspondence. The test proctor shall be:

2.3.2.9.1 Trained by the Contractor in the administration of tests, test security standards, and technology troubleshooting.

2.3.2.9.2 Proficient in English.

2.3.2.9.3 Trained in the administration of the Contractor’s testing system, and able and available to:

2.3.2.9.3.1 Greet test-takers as they arrive.

2.3.2.9.3.2 Check-in and check-out test-takers.

2.3.2.9.3.3 Verify test-taker identity prior to the assessment in accordance with TSA standards.

2.3.2.9.3.4 Print test-related materials and provide them to test-takers.

2.3.2.9.3.5 Monitor test-takers throughout test session, record any incidents (e.g., computer issues, power outages, fire drills, etc.) on a standard incident report form and notify TSA accordingly.

2.3.2.9.3.6 Accommodate any candidate reschedules based on

TSA’s review of incident reports.

2.3.2.9.3.7 Accommodate approved reasonable accommodation

requests (e.g., additional testing time).

HSTS01-15-R-HRM015 Page 16

2.3.2.9.3.8 Be qualified to provide test-takers with assistance throughout their testing appointment including, but not limited to, resolving problems with the test room environment, troubleshooting system problems, rescheduling test-takers when the test center is unable to accommodate a completed test appointment.

2.3.2.9.3.9 Securely dispose of assessment related materials in compliance with guidance set forth by DHS

Management Directive (MD) 11042.1, Safeguarding Sensitive But Unclassified (SBU)

Information.

2.3.2.10 At each testing site, the Contractor shall:

2.3.2.10.1 Require a user authentication: for proctored assessments, a government issued picture identification is required (per requirements established by TSA’s business rules, to be provided post contract award).

2.3.2.10.2 Provide TSA with a daily verified roster and record of attending/non-attending test-takers.

2.3.2.10.3 Have a secure encryption procedure to transmit and receive roster information.

2.3.2.10.4 Have the capability to transmit attendance information and provide incident reports.

2.3.2.10.5 Ensure adherence to test administration protocol.

2.3.2.11 Provide the Contracting Officer Representative (COR) with a Proctor’s

Manual documenting the administration protocol for Contractor-owned/managed facilities, including a supplement to the proctor’s manual that addresses the ADA accommodations. The administration protocol shall address emergency evacuations of the site during the testing session, power failures and computer failures, address ADA related accommodations as outlined by TSA, and matters related to test-takers who are suspected of cheating. The administration protocol shall include procedures for reporting any incident or unusual occurrence during a test session. The administration protocol shall provide a way for the assessment administrator to securely access assessment materials that are

HSTS01-15-R-HRM015 Page 17 unique to each candidate and send candidate information to the Contractor and securely dispose of any paper generated test material.

2.3.2.12 Provide the COR a plan for how the Contractor shall ensure the test administration protocol is followed for Contractor-proctored assessments.

This plan should also address what will occur if a proctor is not present or is late to a testing session. In the event it is necessary to retest a candidate due to failure to follow test administration procedures, the Contractor shall retest the candidate at no cost to TSA.

2.3.2.13 Provide weekend or evening sessions as required by TSA. Testing volume for particular positions may have a larger volume of weekend test administrations and test seat capacity should be commensurate with need.

2.3.2.14 Ensure the Contractor-owned/managed test administration facilities, whether static or mobile, meet acceptable standards for a commercial marketplace computer-based testing environment including but not limited to:

2.3.2.14.1 A secure testing environment that prevents unauthorized users from accessing the testing room while testing is in progress, and that prevents unauthorized users from accessing the computers at any time.

2.3.2.14.2 The site is set up in such a way that the proctor can monitor all test takers either in person or by video at least every 2 – 5 minutes.

2.3.2.14.3 A minimum of five feet exists between test takers, or in situations where 5-feet between test takers is not possible, privacy carrels are utilized and/or computers are positioned such that test takers may not view each other’s responses and/or work.

2.3.2.14.4 A room temperature is maintained (approximately 70-75 degrees).

2.3.2.14.5 Testing rooms and facilities are clean and amenities such as restrooms are nearby.

2.3.2.14.6 Lighting is illuminated to General Services Administration (GSA) standards. (http://www.gsa.gov/portal/content/101308) and glare on computer screens is minimized or eliminated.

2.3.2.14.7 Chairs are sturdy and comfortable.

http://www.gsa.gov/portal/content/101308

HSTS01-15-R-HRM015 Page 18

2.3.2.14.8 Available parking for test-takers and includes handicapped accessible spaces.

2.3.2.14.9 The test center is ADA compliant.

2.3.2.14.10 Bandwidth for all test-takers, internet speed, monitor size, image quality, hardware requirements, sound/headphone requirements, etc. so that a variety of types of assessments can be effectively administered (e.g., video-based simulations, avatar based simulations, multiple choice assessments, etc.).

2.3.2.15 The test administration facilities shall have pens, scratch paper, and highlighters available for candidate use and shall have a functioning printer.

2.3.2.16 Provide the capability to present test takers with feedback/assessment results either immediately or with a delay of up to one hour following each completion of the test or test battery with options to include pass/fail, raw score and percent correct and providing TSA and

Contractor authorized users to select the elements to be presented.

2.3.2.16.1 The time interval between completion of the test/assessment and delivery of the results shall be configurable by test or test battery.

2.3.2.17 A functionally equivalent offline-solution that includes the following:

2.3.2.17.1 Scannable offline forms.

2.3.2.17.2 Distribution of blank offline forms to a TSA centralized location.

2.3.2.17.3 Services for the offline forms to include receiving completed forms, scanning, error checking and attempting to resolve errors

(review forms and log issues), and reporting of logs.

2.3.2.17.4 Create .PDF formatted files for offline test results.

2.3.2.17.5 Upload all possible forms into Web-based CATS and compile with Web-based CATS data.

2.3.2.17.6 Manage retention of all forms in accordance with TSA’s Records

Management requirements (stored in a locked room and/or filing cabinet, retained for 5 years, prepared for and shipped to National

Archives and Records Administration (NARA) facility, etc.).

HSTS01-15-R-HRM015 Page 19

2.3.2.18 Administer un-proctored assessments in accordance with industry standards and best practices, as approved by TSA.

2.3.2.19 Interface with other TSA vendors to send and receive data between the

CATS system and third-party vendor system when the third-party is administering the test/assessment.

2.3.2.20 Schedule Screening Partnership Program (SPP) vendor applicants for taking the CBT.

2.3.2.20.1 Coordinate with SPP Office and the COR for the scheduling of

CBT testing for the SPP vendor applicants.

2.3.2.20.2 Provide SPP applicants’ CBT test results to the SPP Program

Office.

2.3.3 Role-Based Access. The Contractor shall provide a system:

2.3.3.1 With customizable, role-based user access, and incorporate password security and restrictions to all levels of administrator access in accordance with DHS and TSA Information Technology (IT) security standards.

2.3.3.2 With the functionality to perform the following based on the individual’s role-based user access:

2.3.3.2.1 Print tests and test answers keys.

2.3.3.2.2 View and change user information, such as login ID and password.

2.3.3.2.3 Access to test-taker feedback.

2.3.3.2.4 Revise test items.

2.3.3.2.5 Add/delete/edit questions in a test.

2.3.3.2.6 Access all user and testing data.

2.3.3.2.7 Run individual and aggregated reports (e.g., candidates scheduled to test, passing rates, item analysis, distribution of scores, etc.).

2.3.3.2.8 Review tests.

2.3.3.2.9 Assign tests to test takers.

2.3.3.2.10 Add or delete/inactivate users.

HSTS01-15-R-HRM015 Page 20

2.3.3.2.11 Add/modify test taker information to include demographics.

2.3.3.2.12 Send notification messages via e-mail/text.

2.3.3.2.13 Establish date(s) and time(s) for testing.

2.3.3.2.14 Access to test preparation and creation.

2.3.4 Assessment Content Readiness

2.3.4.1 Contractor shall upload, ensure accuracy of content and ensure content is viewable and displayed correctly at the request of TSA and within the timeframes specified below:

2.3.4.1.1 Contractor provided test content – 10 business days.

2.3.4.1.2 TSA provided test content – 10 business days.

2.3.4.1.3 Third-party provided test content – 30 calendar days.

2.3.4.2 Content shall be ready for use immediately after acceptance by TSA.

2.4 Reporting Requirements

The Contractor shall provide a system that can store, report, and transfer information on multiple assessments.

2.4.1 The system shall have the capability to:

2.4.1.1 Be scalable to accommodate multiple assessments and the size of the TSA workforce.

2.4.1.2 Generate TSA approved reports on several thousand cases (e.g., test results for thousands of test takers) at one time.

2.4.1.3 Generate TSA approved reports that reflect item choice data for each test taker to include the correct answer and overall score.

2.4.1.4 Conduct psychometric analyses (e.g., item-level analysis, test-level analysis, etc.) to determine how well tests and test items perform.

2.4.1.5 Support customized reports production (e.g., reference missed, question missed, competency level performance).

2.4.1.6 Report on item banking content across tests.

HSTS01-15-R-HRM015 Page 21

2.4.1.7 Generate real-time TSA approved predefined reports; TSA approved aggregate reports for subsets of data; and TSA approved ad hoc reports based on pre-existing data elements captured in the system (e.g., time-based reporting/comparisons, such as quarter to quarter, year to year;

results by airport).

2.4.1.8 Upload and transfer candidate assessment results automatically into the

TSA hiring systems within 12 hours of assessment completion. The content of the data transferred will be determined after contract award

(e.g., pass/fail status, percentile based on established test norms, raw score, etc.).

2.4.1.9 Provide designated TSA employees access to pull reports and data as necessary.

2.4.1.10 Provide the capability to export data and reports in a generic file format such as raw data in delimited ASCII format or other agreed-upon formats.

2.4.1.11 Encrypt all sensitive data transmitted outside system boundaries using

FIPS 140-2 validated cryptographic modules, and algorithms compliant with FIPS 197 (AES 256).

2.4.1.12 Enable TSA access to current and historical assessment results, to be determined after award of contract based on TSA business rules. The

Government may require the Contractor to turn specific candidate scores on or off within two (2) business days of notification. TSA can request that the Contractor turn on or off specific candidate scores when necessary.

2.4.1.13 Capture test administration data including but not limited to responses, item latencies, assessment start time, and assessment end time.

2.4.1.14 Store assessment and test data and results, in compliance with TSA’s requirements for the safeguarding of Personally Identifiable Information

(PII) and SSI, through the end of the contract.

2.4.2 The Contractor shall provide the TSA Program Manager (PM) and/or their designee with aggregated and individual tests assessment results within two business days of the

Government’s electronic request.

HSTS01-15-R-HRM015 Page 22

2.5 Assessment Library

2.5.1 The Contractor shall provide TSA with continuous, electronic access to the list of tests and assessments included in their Assessment Library along with a brief description of the test/assessment including but not limited to:

2.5.1.1 Constructs and/or competencies that the test was designed to measure.

2.5.1.2 Test format (e.g., multiple choice, video based simulation, computer adaptive).

2.5.1.3 Test administration method (e.g., proctored, un-proctored).

2.5.1.4 Test length (e.g., number of items, number of scenarios or events if a simulation) and the amount of time allowed to complete the test (for untimed tests the average time to complete the test).

2.5.1.5 Purpose of the test, if available.

2.5.1.6 Pricing.

2.5.2 Upon request from authorized TSA personnel the Contractor shall provide the Test

Manual or Technical Report that fully describes the development and validation of the test, the psychometric properties of the test (e.g., reliability and validity information), and definitions of the constructs and/or competencies that the test was designed to measure.

2.5.3 Upon request from authorized TSA personnel, the Contractor shall assist TSA in locating third-party assessments. Once located, the Contractor shall provide TSA with a brief description of the test and an overview of its psychometric properties.

2.5.4 Once TSA has selected and implemented an assessment from the assessment library, the

Contractor shall ensure that assessment remains in the library for use throughout the life of the contract.

2.6 Assessment Development

2.6.1 Upon request from TSA, the Contractor shall develop custom assessments in order to evaluate the degree to which an individual possesses job-related, technical or non-technical competencies necessary to perform effectively in a specific position or a group of like positions. The Contractor shall:

2.6.1.1 Develop a draft Assessment Development Project Plan that shall include a project timeline describing the tasks and activities that will be accomplished. The draft Project Plan shall be due at the Task Order

HSTS01-15-R-HRM015 Page 23

Award Kickoff meeting. TSA will provide the Contractor with review comments within 10 business days after the draft Project Plan is received.

The Final Project Plan shall be provided to the COR and PM within 10 business days from the date the review comments were submitted to the

Contractor.

2.6.1.2 Develop all materials required to administer and score the assessment(s), such as: test administration instructions, test taker instructions, background materials, scoring protocols, evaluation guidelines or answer keys, and any other materials that are required to fully implement the assessment(s).

2.6.1.3 Provide TSA with an initial draft of each assessment for the Government’s review and acceptance per the specified timeline in the Government approved project plan. The Government will respond to the Contractor’s initial draft assessments per the specified timeline in the Government approved project plan.

2.6.1.3.1 Incorporate TSA’s feedback and submit a final version per the specified timeline in the Government approved project plan.

2.6.1.4 Incorporate the capability for an automated feedback report to be generated for each test taker completing the assessment.

2.6.1.4.1 This feedback report shall display a variety of feedback and performance dimensions as specified by TSA, depending on the assessment and position (e.g., competency feedback, percentile score, generic automated developmental comments).

2.6.1.5 Upload the assessment(s) and all associated materials into the CATS system once approved by TSA and in accordance with section 2.3.4

Assessment Content Readiness.

2.6.1.6 Content-validate the assessment in accordance with the Equal

Employment Opportunity Commission (EEOC) Uniform Guidelines for

Employee Selection (1978), the Society for Industrial and Organizational

Psychology (SIOP) Principles for the Validation & Use of Personnel

Selection Procedures 4 th

Edition (2003), and the Standards for

Educational & Psychological Testing (1999).

2.6.1.7 As needed, maintain and/or make minor updates and corrections to the assessment once it is in use as issues and/or concerns are identified.

HSTS01-15-R-HRM015 Page 24

2.7 Item Development

2.7.1 The Contractor shall develop, maintain, and/or update test items used for initial certification and/or recertification testing.

2.7.2 The Contractor shall develop a draft Item Development Project Plan that includes a project timeline describing the tasks and activities that will be accomplished. The draft

Project Plan shall be due at the Task Order Award Kickoff meeting.

2.7.3 The Contractor shall review TSA provided items to ensure alignment with current SOPs and other source material. Contractor shall provide an Impact Analysis of edits that may have to be made and identify sections that do not have previously developed items. If needed, the Contractor shall edit items and submit all test items to TSA for approval.

2.7.4 As necessary, the Contractor shall provide new test items based upon TSA specifications and in accordance with TSA’s Item Writing Guidelines.

2.7.5 All test items shall be entered into CATS and submitted to TSA for review in batches of no more than 50 items. An excel test map shall also be provided.

2.7.6 Revisions and updates to the test items may be necessary based on any TSA Subject

Matter Expert (SME) reviews, pilots, and or source material revisions.

2.8 Validation Studies

2.8.1 The Contractor shall have the ability to conduct a variety of validation studies (e.g., criterion-related, construct, transportability, etc.) or assist TSA with same.

2.8.2 The Contractor shall conduct validation studies in accordance with the EEOC Uniform

Guidelines for Employee Selection (1978), the SIOP Principles for the Validation & Use of Personnel Selection Procedures (2003) and the Standards for Educational &

Psychological Testing (1999).

2.8.3 The Contractor shall develop or obtain materials required to collect the data for the validation study including but not limited to:

2.8.3.1 Developing forms for collecting performance data for research purposes only.

2.8.3.2 Working with TSA to identify organizational data suitable for use as criterion data.

2.8.3.3 Identifying assessments to be used as part of a construct validation study, including assessments used to establish concurrent and discriminant

HSTS01-15-R-HRM015 Page 25 validity.

2.8.4 The Contractor shall provide TSA with all forms developed and used as part of the validation study at the conclusion of the study and shall destroy any copies in their possession whether paper or electronic.

2.8.5 The Contractor shall provide TSA with all data collected as part of the validation study.

The data shall be provided in a readily useable format agreed upon…