Attachment 4C - OTSS RFQ Attachment OTSS-4 - Statement of Work - A0001 - Clean.pdf

Attachment OTSS-4 – Statement of Work

1.0 Overview

This document defines the services needed to provide the Transportation Security Administration (TSA), with Operational Test and Evaluation (OT&E) activities, agency wide field pilot/demonstration, data collection and analysis services. These services are referred to as Operational Test and Evaluation Support Services (OTSS).

As provided for in Task Orders, the Contractor shall provide all management, labor, travel, expertise, and logistics for all Operational Test Branch (OTB), System Evaluation Branch (SEB) evaluation and liaison activities including but not limited to the following:

a. Program Management activities including, but not limited to, resource allocation, reporting, scheduling, planning, data management, test logistics, shipping, and any “in office” functions such as clerical or administrative support functions.

b. Technical Support activities including all developmental activities that enable test planning, test execution, development of test articles, verification of test articles, software development for data capture, software development for test analysis, engineering change request preparation and review.

c. Test Execution including the execution, assessing and reporting of all operational testing and field data collection activities.

d. Personnel, including providing subject matter experts, analysts, statisticians, test leads, and administration, as applicable, to support the Blanket Purchase Agreement (BPA)PA and any subsequent Orders.

e. Operational data collection including operational utility evaluation, operational test, stream of commerce documentation, cargo system qualification, canine (K9) training/testing, maintenance, covert performance testing, and tracking of all test equipment. Tracking of test equipment shall include documenting inspections, maintenance, and disposition of test equipment.

f. Technical and Analytical support activities for Test and Evaluation strategy development to include integrated test concepts and operational test measures to support evaluation of program operational requirements and Key Performance Parameters (KPPs).

g. Technical and Analytical support activities for System Evaluation reporting, to include statistical processing of test data, establishing cause-and-effect of observed system performance, documenting a systems’ performance against its Operational Requirements and KPPs, and producing evaluations of Effectiveness, Suitability and Resilience.

1.1 General Background

The TSA is charged with the mission of protecting the nation’s transportation systems to ensure freedom of movement for people and commerce. The mission of Acquisition Program Management (APM) is to develop and implement the best procedures and technologies that deter, prevent, and/or render ineffective any attempt to sabotage commerce and transportation. APM’s Test and Evaluation Division (TED) provides test, evaluation and lifecycle matrix support services to APM program offices, TSA field elements and other TSA and Department of Homeland Security (DHS) stakeholders. The TED Test and Evaluation (T&E) group conducts timely, efficient, and robust T&E in support the TSA’s mission by providing efficient and effective security technology and capability solutions through applied research, development, operational testing, and life cycle management to ensure the free movement of people and commerce. As a DHS approved Independent Test Agent (ITA), the TED T&E group primarily conducts OT&E of TSA Transportation Security Equipment (TSE) acquisitions, and may conduct OT&E for other TSA and DHS acquisition programs. TED is the primary liaison with DHS Science & Technology (S&T) on T&E matters, to include T&E policy level coordination. TED is also responsible for the following:

a. Management of all technical and logistic activities associated with the TSA Systems Integration Facility (TSIF);

b. Serving as the Independent Operational Test Agent (IOTA) for acquisition systems and Investment Review Board (IRB) decisions; and

c. Developing and maintaining a T&E capability to support all testing within TSA to include acquisition support, Concept of Operations (CONOPS), process development/refinement, and threat mitigation.

The TED Director is responsible for executing and maintaining the APM/Component Acquisition Executive (CAE) T&E Policy document, as well as the APM T&E Evaluation Guidebook.

The primary objective of OT&E is to validate that a new or modified system is operationally effective, suitable, and resilient for use by TSA, and that the infrastructure is ready to accept the system. These tests focus on evaluating whether KPPs and Operational Requirements are met and that all Critical Operational Issues (COIs) are resolved. TSA conducts operational testing at field sites using field-representative personnel with production-representative systems. OTSS includes, but is not limited to, OT&E activities and services such as Operational Testing (OT), Operational Utility Evaluation (OUE), Operational Assessment (OA), Test and Evaluation Strategy and Planning, Integrated Testing, as well as other field-based test or data collection activities such as reliability data collection, covert performance testing, field pilot tests, procedure trials, and baseline operational data collection. OTSS also includes supporting test article development and logistics for a variety of field test activities.

1.2 Technical Documents and Requirements

The Contractor shall adhere to the requirements of the following attachments which are included as part of the BPA.

a. Department of Homeland Security (DHS) Directive Number 102-01, Acquisition Management Directive, Revision 03, July 2015;

b. DHS Directive Number 026-06, Test and Evaluation, Revision Number 01, May 2017;

c. DHS T&E Management Guide, Version 3.0, January 2019;

d. DHS Supplemental Guidance, Threat Assessment in Support of T&E, Version 1.0, September

2018;

e. DHS Cyber Resilience T&E, T&E Supplemental Guidance, Version 2.0, October 2019;

f. TSA T&E Guidebook, Revision 5, June 2022;

g. TSA T&E Policy, Revision 3, August 2018;

h. TSA Management Directive 2600.4, Test and Evaluation;

i. TSA T&E, Verification, Validation, and Accreditation (VV&A) Policy, May 2014;

j. TSA T&E Statistical Methodology Policy Memorandum, April 2016.

Copies of the Technical Documents are available upon request.

1.3 Sensitive Security Information and Security Clearances

Much of the information that the Contractor will encounter during the performance of the BPA is Sensitive Security Information (SSI). All Contractor personnel are required to receive a favorable TSA suitability determination prior to commencing work on the contract. All material marked SSI shall be protected via TSA approved procedures.

As provided in Section 1.4, TSA will provide Government Furnished Equipment (GFE), such as laptops, and drives, and issue Personal Identity Verification (PIV) credentials as well as TSA email accounts to handle SSI information. The Contractor shall input, store, process, and/or transmit sensitive information using GFE. Any processing of Sensitive but Unclassified (SUB) data into classified information shall take place on Homeland Secure Data Network (HSDN) terminals at TSA provided secured facilities.

For some Task Orders, specific personnel will require access to classified information (at a minimum of a SECRET level) to perform the work; however, TSA does not anticipate requiring the handling and storage of classified information at the Contractor’s facility. TSA will provide access to TSA’s classified processing and storage facilities on an as needed basis. All individuals that possess a valid need to know and/or will be working on key positions that require a classified clearance or handle classified information shall be required to be cleared at the appropriate clearance level. Information regarding the Task Oder’s classification level will be provided with the Task Order Request for Quote (RFQ).

The Technical Monitor (TM) and/or Contracting Officer Representative (COR) will determine if an individual will require access to classified material and a SECRET clearance. TSA will direct the Contractor to initiate a security clearance investigation based on that determination via a formal notification from the COR or TM.

1.4 Government Furnished Equipment (GFE)

TSA will provide GFE, such as laptops, drives, and data collection tools and will also issue PIV credentials as well as TSA email accounts to handle SSI information.

The Government anticipates providing approximately 10 laptops to support the test execution tasks, 30 laptops to support the test planning, analysis, reporting tasks, and 10 laptops to support the TSA strategic planning and coordination support tasks for a total of approximately 50 laptops. The Government will provide additional laptops on an as needed basis when approved by the COR or TM.

TSA will provide data collection tools (e.g. gauges, instruments, measurement equipment, data recording equipment, electronic tools, camera, necessary software or hardware), and test articles (e.g.

test objects, test bags, threat inject articles). TSA will also provide test supplies and consumables needed to support the test event (e.g. materials to assemble or repair test articles, replacement parts for tools, safety gear and supplies).

1.5 Data Management

The Contractor shall collect all test data using the Government approved data format and store this data in Government-approved or Government specified locations and storage mediums.

The Contractor shall maintain a test and evaluation database on the TSA network. All test data shall be stored in GFE. Currently, test team final documentation are uploaded to a TSA-provided file storage system on iShare and network storage. The Contractor shall coordinate with TSA organizations responsible for maintaining these systems to maintain and expand TSA’s utilization of these systems.

The Contractor shall obtain approval from the COR or TM prior to implementing any data management solution. In the event that an Authority to Operate (ATO) is required, the Contractor shall provide the ATO at no additional cost to the Government.

2.0 Test Execution (Basic, Intermediate and Advanced Data Collection Units)

2.1 General

The Contractor shall provide Basic, Intermediate, and/or Advanced Data Collection Units to support test events. The number and type of the Data Collection Units to be provided by the Contractor will be specified by the Government in the evaluation strategy/System Evaluation Plan (SEP) and detailed in the Operational Test Plan (OTP), Operational Assessment Plan (OAP) or Operational Utility Evaluation Plan (OUEP) and will be provided to the Contractor at the order level.

Test Planning, Analysis and Reporting (PAR) and Program Management activities may also be required for tests. These activities are covered by separate tasks.

For tracking and invoicing purposes, TSA has organized data collection units into distinct tasks by the general type of equipment being tested or program supported as provided in the chart below:

TASK NUMBER TASK DESCRIPTION

CLIN X001 Checkpoint Operational Test Support

A-1 Basic Data Collection Unit A-2 Intermediate Data Collection Unit A-3 Advanced Data Collection Unit

CLIN X002 Checked Baggage Operational Test Support

A-7 Basic Data Collection Unit A-8 Intermediate Data Collection Unit A-9 Advanced Data Collection Unit

CLIN X003 RCA/ITF Test Support A-13 Basic Data Collection Unit A-14 Intermediate Data Collection Unit A-15 Advanced Data Collection Unit

CLIN X004 RCA Rates and Standards, Modeling and Simulation Support

A-19 Basic Data Collection Unit A-20 Intermediate Data Collection Unit A-21 Advanced Data Collection Unit

CLIN X005 Information Systems Test Support A-25 Basic Data Collection Unit A-26 Intermediate Data Collection Unit

A-27 Advanced Data Collection Unit CLIN X006 Cybersecurity Test Support

A-31 Basic Data Collection Unit A-32 Intermediate Data Collection Unit A-33 Advanced Data Collection Unit

Basic Data Collection personnel shall be able to conduct direct observations near TSE, screening officers and/or passengers, and record information for up to eight and one half (8.5) hours in the screening environment (i.e. screening checkpoint or checked baggage areas). One Basic Data Collection Unit is one

(1) Basic Data Collector for one (1) 8.5 hour test day.

Intermediate Data Collection personnel shall serve as Site Leads for each test site as defined in the test plan for the test event. Intermediate Data Collection personnel shall be able to conduct direct observations, record information and supervise a team of Basic Data Collection personnel for up to eight and one half (8.5) hours in the screening environment. One intermediate Data Collection Unit is one (1) Intermediate Data Collector for one (1) 8.5 hour test day.

Advanced Data Collection personnel serve as Subject Matter Experts (SMEs) as defined in the test plan for the test event. Advanced Data Collection personnel may include Human Factors Engineers, Information Assurance Specialist, or other SMEs to support the evaluation. Depending on the evaluation strategy and operational test plan, the Advanced Data Collection personnel may support a baseline, burn-in test, or other type of test event when the measures associated with the test event require their expertise. One Advanced Data Collection Unit is one (1) Advanced Data Collector for one (1) 8.0 hour test day.

The mix of Basic, Intermediate, and Advanced Data Collection Units will be specified at the order level.

The following general requirements apply to Basic, Intermediate and Advanced Data Collection Units:

The Contractor shall provide sufficient training for Basic, Intermediate and Advanced Data Collectors to ensure that the data collectors have a clear and concise understanding of the test plan, data collection tools, screening Standard Operation Procedures (SOPs), screening technologies, screening processes and related information prior to the start of data collection on site.

The Contractor shall provide personnel to support simultaneous test activities occurring in multiple locations with a sufficient cleared/vetted bench of personnel to allow for ramp-up within one week after receiving written notice or an order from the Government. Based on the estimated required testing, it is estimated that upward of 100 test resources will be required in the field conducting test activities simultaneously at multiple airports/sites.

Please note that Data Collectors will not generally be provided with a TSA laptop, rather, they will record their observations either on paper or using other data collection tools provided by the Government.

All associated travel costs and travel hours must be authorized in advance by the COR and/or TM.

Travel labor hours will be accounted for in Data Collection Units.

The Government will attempt to provide advanced notice of upcoming testing where possible;

however, the Contractor shall be prepared to support any given test event/site with one (1) weeks’ notice.

Please note: During the Transition-In Period, the Government will attempt to provide additional notification.

The Government has found that using Data Collectors who are not familiar with the system under test increases the risk of introducing errors into the test process and lowering the confidence in test data and test results. Therefore, the Contractor shall maximize the use of Data Collectors who are already familiar with the technology being tested, the SOP, and environment the system is in.

The same set of Data Collectors shall support all phases of the test event to include baseline, burn-in, and record testing to ensure continuity, familiarity and consistency with the test procedures and the test environment.

The Government may require that the Contractor use video data collection for future data authentication purposes to determine true uptime, downtime, incident documentation, and causality investigation. The purchase of video data collection tools would be Other Direct Costs (ODC.).

Intermediate Data Collectors (i.e. Site Leads) shall prepare daily written summaries of each event. In the case of the Contractor conducting the same data collection at multiple sites, the Contractor shall provide a consolidated report including a summary for each site. [Deliverable D.14]. These daily summaries may contain SSI.

Data Collectors shall exhibit a low operational foot print during data collection on site and on duty.

Data Collectors must remain as unobtrusive as possible, and should only engage end users (i.e., Transportation Security Officers), or other end users of the system under evaluation when necessary to support the initiative (i.e., clarify an activity or scan result that the data collector missed).

The Contractor shall comply with Government stakeholder’s safety, security, and logistics protocols to protect test integrity, to prevent safety incidents, loss, or theft of the test and threat articles, or inadvertent disclosure of test procedures and materials.

The Contractor shall apply all appropriate measures to safeguard test data and results. The Contractor shall process test data that may identify security vulnerabilities which are SSI or classified in accordance with TSA policy.

The Contractor shall provide operational flexibility and utilize a staffing model to insure that resources are available to support test site (i.e., airport) events with variable test start/finish dates that may require adjustments until the start of the test. The deployment of new products /services /systems is inherently a dynamic and complex process that often results in program schedule adjustments for a large variety of reasons.

Airport procedures may require pre-event visits to the test sites to obtain necessary badges or credentials. Additionally, pre-event visits may be required to attend test specific training.

Pre-event visits (and associated travel costs) must be authorized in advance by the COR and/or TM.

The Government intends to utilize Data Collection units for pre-event site visits with travel costs (if any) being reimbursed under the ODC CLIN.

Contractor employees working at an operational test site shall be responsible for obtaining, possessing, and displaying an appropriate badge in accordance with the airport’s physical and personnel security requirements.

If a Contractor replaces a badged test team member, the Contractor shall be responsible for all costs associated with the badging of that team member’s replacement.

2.2 Threat Inject Test Execution

Test execution events will typically include Threat Inject (TI) testing. TSA conducts approximately 3,000+ TI trials per year in an attempt to penetrate airport security across a large variety of operational environments to evaluate the detection capability of systems under test. TI data collection activities are executed under realistic conditions using threat articles or procedure for the purpose of determining the security or operational effectiveness of the equipment, process or system. Threat Inject testing is performed during the record test phase of an event.

In addition to Threat Inject Test Execution, the Contractor shall also be responsible for inventory management, shipping, and control of all test articles (to include threat inject articles and supplemental equipment) used in support of the TI test events. For events taking place in passenger screening checkpoint environments, and/or focused on identity management technology testing, the Contractor shall produce realistic travel documents and identification cards, both paper and mobile format for use during the test by TI carriers. TSA will provide relevant information to inform the development of all test articles for the test events. Typically, the inventory management, shipping and storage of test articles are covered under the ODC Task while the creation of realistic travel documents is covered under the PAR Task activities for the test.

The following general requirements apply to TI Execution:

All TI carriers shall have a minimum of a SECRET Clearance.

The Contractor shall provide Basic Data Collectors trained to serve as TI carriers. Generally, one (1) to twelve (12) TI carriers shall be required per test event across multiple sites.

TI carriers shall be both male and female with varying body types and ages (minimum 18 years old) for the test event.

TI carriers must have the ability to perform through an inflexible script as well as on an impromptu basis to carry the TI test article in accessible properties, on person or in checked baggage as defined in the test plan. The TI carrier shall be able to “live” a role and freely adapt to it in response to the actions of screening officer. If this adaptation is not completed, the immediacy and realism of the TI trial may be compromised.

One TI carrier is one (1) Basic Data Collector for one (1) 8.5 hour test day.

The Contractor shall be able support threat inject execution across all test sites simultaneously.

TI carriers shall be rotated across test sites for standalone checked baggage, checkpoint, and ITF test events to reduce variability in personality, presentation, mannerisms, identification, etc., that could affect test outcome.

The Contractor shall conduct TI testing covertly utilizing test articles, provided by the Government, comprised of live explosives, explosive simulants, prohibited items, or other items or methods as defined in the TI test plan.

The Contractor shall provide Intermediate Data Collection personnel to serve as TI Site Coordinators for each test site as defined in the test plan for the test event. The TI Site Coordinator shall lead the coordination of TI carriers, manage the TI test article, ensure TI trials are executed in accordance with the test plan. The TI Site Coordinator shall coordinate with the Data Collection Site Lead to collect TI data throughout all the TI trials. One TI Site Coordinator is one (1) Intermediate Data Collector for one

(1) 8.5 hour test day.

The Contractor, along with the Federal test team, shall collaborate closely with Government stakeholder’s safety, security, and logistics protocols to protect test integrity, to prevent safety incidents, loss, or theft of the test and threat articles, or inadvertent disclosure of test procedures and materials.

The Contractor, along with the Federal test team, shall coordinate all TI tests with appropriate HQ TSA staff, local management, local test site and Law Enforcement officials to eliminate inadvertent Law Enforcement responses to TI test scenarios.

As part of the PAR Task, the Contractor shall coordinate the logistics and shipping of test articles from federal locations to airport test locations. The Contractor shall arrange for on-site secure storage at identified test sites for all data collection tools and test articles. The Government currently does not have predesignated on-site/off-site secure storage locations/facilities pre-determined or contracted. As part of the PAR Task, the Contractor, in coordination with TSA, will identify suitable storage locations/facilities, usually at airport test sites. If it is identified that the airport is unable to provide storage at no-cost, storage costs will be covered under the ODC Task. All storage costs must be authorized in advance by the COR and/or TM.

The Contractor shall inspect the data collection tools, test articles, and threat inject test articles for damage before and after use at each test location. Threat inject test articles damaged to an extent that hinder test results or prevent safe use during testing shall be documented and sent to the Government facility for maintenance and revalidation actions.

During test execution, no TI carrier or TI items may visit the same checkpoint or standalone checked baggage system more than twice in a day. Ideally no checkpoint or standalone checked baggage system should be subjected to more than three (3) injects on any given day. Every attempt will be made to have the majority of the TI carriers conform in appearance with the predominant demographic of the targeted screening population. TSA designs these constraints by limiting the numbers of threat inject test scenarios to prevent test compromise or artificially raise the general awareness or alert level of the screening workforce. Careful monitoring of the results as testing proceeds is required and may drive modifications to the Threat Inject test scenario schedules. The Government OT Director must approve any deviations.

The following sections 2.3 – 2.8 provide information on the general categories of testing as well as the estimated number of tests.

2.3 Checkpoint Operational Test Support (Tasks A-1, A-2, and A-3)

TSA’s Checkpoint Operations include K9 screening, identity management and vetting, accessible property screening, on-person screening, alarm resolution and other field information systems related to passenger or accessible property security screening. Please note that K9 screening is separately covered within this SOW.

The Government conducts test events for two (2) general categories of Checkpoint TSE: Large and Small scale systems. Large scale systems include sizable imaging systems, such as but not limited to, accessible property Computed Tomography (CT) scanning or Advanced Imaging Technology (AIT) for on-person screening. Small scale systems include but are not limited to Explosive Trace Detection (ETD) systems, Credential Authentication Technology (CAT) systems, and Bottle Liquid Screening (BLS) systems. Small scale systems are designed to fit onto or nearby the current podiums, tables, or other fixtures, and possibly with their own kiosk or cart.

Checkpoint Technology (CPT) Operational Test Support includes three (3) types of test events: OT&E, Follow-on OT&E (FOT&E), and OUE.

The Government estimates that the following number of CPT test events / activities will be required per year:

Test Execution Event or Data Collection Support Activities

Estimated Number of Events /Activities

Estimated Number of Baseline Days

Estimated Number of Burn-in Days

Estimated Number of Record Days

CPT Operational Test Support

CPT OT&E 10-15 0-20 5-10 20-30

CPT FOT&E 10-15 0-20 5-10 20-30

CPT OUE 10-15 0-10 2-10 10-21

Program requirements drive the number and scope of CPT events. A single TSE evaluation may include systems from multiple vendors, various configurations, and variation of screening concepts, process and procedures.

An OT&E or a FOT&E of a vendor-system or a configuration may include one (1) to three (3) airport test sites with one or more test units at each site. Testing may include baseline testing scheduled for up to twenty (20) days, which the Government typically performs prior to burn-in and record testing. TSA may schedule burn-in testing (5-10 days), but configuration changes, Standard Operating Procedures (SOP) modifications, or upgrades in training can extend the schedule. The Government schedules record test (20-30 days), which is based upon the scope of requirements under test for that event. The Government deploys a test team per test site. A test team may consist of five (5) to ten (10) Basic Data Collectors to conduct direct observations and record data, two (2) Intermediate Data Collectors to serve as Site Leads (or Shift Leads when testing continues to the second shift) to manage the team and record data, and an Advanced Data Collector may serve as Subject Matter Expert (SME) on site for a duration of one (1) to five (5) days.

An OUE of a vendor-system or a configuration may include one (1) to three (3) airport test sites with one

(1) test team at each site. Testing may include baseline testing schedule for up to ten (10) days, which the Government typically performs prior to burn-in and record testing. The Government may schedule burn-in testing (2-10 days), but configuration changes, SOP modifications, or upgrades in training can extend the schedule. TSA schedules record test (10-21 days), which is based upon the scope of requirements under test for the event. The Government deploys a test team per test site. A test team may consist of two (2) to five (5) Basic Data Collectors to conduct direct observations and record data and one (1) Intermediate Data Collector to serve as Site Lead to manage the team and record data.

2.4 Checked Baggage Operational Test Support (Tasks A-7, A-8, and A-9)

APM’s Checked Baggage Technologies (CBT) Division (CBTD) performs planning and execution of acquisition, procurement, and strategic planning activities for the Electronic Baggage Screening Program. CBTD manages and prepares equipment recapitalization plans, ensures compliance with acquisition guidelines and policy, oversees planning an execution of budge activities, and coordinate with the DHS Science and Technology (S&T) on research and development (R&D) and detection activities. The CBT Division ensures that the most technologically efficient and effective capabilities and technologies are available for screening checked baggage.

TSA provides test and evaluation support services for the following CBT systems:

a. In-line Explosives Detection System (EDS) are those systems TSA designed to integrate in a Baggage Handling System (BHS) and usually networked with other systems sharing primary and secondary viewing stations.

b. Standalone EDS units are fully self-contained systems usually associated with lobby applications and may contain one or more primary and secondary viewing station.

c. EDS with Security Technology Integrated Program (STIP) integration. As background, STIP designed to facilitate the connection of screening equipment. This may include but, is not limited to integration testing of STIP-TSE Interface Requirements Document and other STIP requirements in the TSE ORD and FRD.

Checked Baggage Technology (CBT) Operational Test Support includes three (3) types of test events:

OT&E, FOT&E, and OUE.

The Government estimates that the following number of CBT test events / activities will be required per year:

Test Execution Event or Data Collection Support Activities

Estimated Number of Events /Activities

Estimated Number of Baseline Days

Estimated Number of Burn-in Days

Estimated Number of Record Days

CBT Operational Test Support

CBT OT&E 1-3 0-20 5-15 30-45

CBT FOT&E 1-3 0-20 5-15 20-45

CBT OUE 1-3 0-10 2-10 10-21

Program requirements drive the number and scope of CBT or CBT/STIP events. A single TSE evaluation may include systems from multiple vendors, various configurations, and variation of screening concepts, process and procedures.

An OT&E or a FOT&E of a vendor-system or a configuration may include one (1) to three (3) airport test sites with one or more test units at each site. Testing may include baseline testing scheduled for up to twenty (20) days, which TSA typically performs prior to burn-in and record testing. The Government may schedule burn-in testing (5-15 days), but configuration changes, SOP modifications, or upgrades in training can extend the schedule. The Government schedules record test (30-45 days), which is based upon the scope of requirements under test for that event. The Government deploys a test team per test site. A test team may consist of five (5) to ten (10) Basic Data Collectors to conduct direct observations and record data, two (2) Intermediate Data Collectors to serve as Site Lead (or Shift Lead) to manage the team and record data, and an Advanced Data Collector may serve as SME on site for a duration of one

(1) to five (5) days.

In-line EDS threat inject testing does not typically involve separate threat inject carriers at each test site due to the nature of the airport checked baggage layout and configurations. Rather, threat inject testing is typically managed by the data collection staff for all in-line systems.

An OUE of a vendor-system or a configuration may include one (1) to three (3) airport test sites with one

(1) test units at each site. Testing may include baseline testing schedule for up to ten (10) days, which The Government typically performs prior to burn-in and record testing. The Government may schedule burn-in testing (2-10 days), but configuration changes, SOP modifications, or upgrades in training can extend the schedule. The Government schedules record test (10-21 days), which is based upon the scope of requirements under test for the event. The Government deploys a test team per test site. A test team may consist of two (2) to five (5) Basic Data Collectors and one (1) Intermediate Data Collector.

2.5 Requirements and Capabilities Analysis/Innovation Task Force Test Support (Tasks A-13, A-14, and A-15)

TSA applies a variety of technological and physical security measures, methods and systems for screening, and is continuously looking to improve performance and efficiency. The Innovation Task Force (ITF) fosters innovation by identifying and demonstrating emerging solutions to improve security effectiveness, efficiency, passenger experience, flow of commerce, and deliver requirements or solutions. Since inception in 2016, ITF has demonstrated the Automated Screening Lanes (ASL), CT scanner, enhanced AIT, new biometric capabilities, Ultraviolet C (UVC) sanitation technologies, as well as other novel advancement of signage and training. Every year ITF announces the Innovative Demonstrations for Enterprise Advancement (IDEA) Targeted Broad Agency Announcement (BAA) to gather inputs from Industry and selects solutions for demonstration.

The Requirements and Capabilities Analysis (RCA)’s Capability Managers (CMs) develop the capability roadmaps and drive the requirement for development and early testing of emerging technologies.

TSA provides field evaluation and test support services to RCA CMs and ITF. There are two (2) general types of test activities expected under the ITF Tasks; test observation and operational test data collection. TSA defines test observation as the test activities that the ITA is not primarily responsible for leading, planning and executing; rather the Contractor shall participate in the activities to provide inputs to planning, conduct onsite observation and oversight to capture data that can be utilized for the overall system evaluation. Operational test data collection is all the data collection activities where the TED and this Contractor are responsible for planning, executing, analyzing and reporting. TSA will develop specific data collection requirements for each event as planning continues. Test objective will drive the requirements and will include, but are not limited to, test site location, number of data collectors, applicable TI scenarios, and test duration.

A RCA/ITF OUE to support test observation, the Government deploys a test team per test site. A test team may consist of two (2) to five (5) Basic Data Collectors and two (2) Intermediate Data Collectors.

An Advanced Data Collector may serve as SME on site for a duration of one (1) to five (5) days.

A RCA/ITF OUE to support operational test data collection, the Government deploys a test team per test site. A test team may consist of five (5) to ten (10) Basic Data Collectors and two (2) Intermediate Data Collectors. An Advanced Data Collector may serve as SME on site for a duration of one (1) to five (5) days.

RCA/ITF Test Support includes three (3) types of test events: Type 1, Type 2, and Type 3 OUEs.

The Government estimates that the following number of RCA/ITF test events / activities will be required per year:

Support Activities

Estimated Number of Events /Activities

Estimated Number of Baseline Days

Estimated Number of Burn-in Days

Estimated Number of Record Days

RCA/ITF TEST SUPPORT

OUE Type 1 1-3 0-10 2-10 22-30 OUE Type 2 1-3 0-10 2-10 15-21 OUE Type 3 1-3 0-10 2-10 7-14

Program requirements drive the number and scope of RCA/ITF events. A single TSE evaluation may include systems from multiple vendors, various configurations, and variation of screening concepts, process and procedures.

A RCA/ITF OUE of a vendor-system or a configuration may include one (1) to three (3) airport test sites with one or more test units at each site. Testing may include baseline testing scheduled for up to ten

(10) days, which The Government typically performs prior to burn-in and record testing. The Government may schedule burn-in testing (2-10 days), but configuration changes, SOP modifications, or upgrades in training can extend the schedule.

A RCA/ITF OUE Type 1 test event may include record test for twenty-two (22) to thirty (30) days, which is based upon the scope of requirements under test for the event.

A RCA/ITF OUE Type 2 test event may include record test for fifteen (15) to twenty-one (21) days, which is based upon the scope of requirements under test for the event.

A RCA/ITF OUE Type 3 test event may include record test for seven (7) to fourteen (14) days, which is based upon the scope of requirements under test for the event.

2.6 RCA Rates and Standards, Modeling and Simulation Support (Tasks A-19, A-20, and A-21)

TSA conducts periodic field assessment of checkpoint and checked baggage screening performance to inform the annual screening performance rates and validate assumptions used in the resource allocation planning standards for screening workforce budget planning model.

Annually, Rates and Standards (RS) Support includes four (4) types of support activities: Type 1 OUE, Type 2 OUE, Type 3 OUE, and Modeling and Simulation (M&S) event.

The Government estimates that the following number of RS test events / activities will be required per

Support Activities

Estimated Number of Events /Activities

Estimated Number of Baseline Days

Estimated Number of Burn-in Days

Estimated Number of Record Days

RCA Rates and Standards, M&S Support OUE Type 1 25-30 0 0-2 5-12 OUE Type 2 15-20 0 0-2 3-5 OUE Type 3 10-15 0 0-2 1-3

RCA/MS 0-2 - - -

Program requirements drive the number and scope of RS events. A single TSE evaluation may include systems from multiple vendors, various configurations, and variation of screening concepts, process and procedures.

A Type 1 OUE event includes one (1) airport test site with one or more screening systems (AT, CT, AIT, etc.) and multiple screening configurations (Standard vs. TSA Precheck) during a single event. It is critical that data collectors are familiar with TSA screening technologies, processes and procedures. Data collection includes up to 2 days of burn-in and five (5) to twelve (12) days of record testing. A test team may consist of five (5) to seven (7) Basic Data Collectors and one (1) Intermediate Data Collector. A test team will travel from one event to another event consecutively for up to 12 events.

A Type 2 OUE event includes one (1) airport test site with one or more screening systems (CAT, alarm resolution, bag search at EDS, passenger arrival, etc.) and multiple screening configurations (Standard

vs. TSA Precheck) during a single event. It is critical that data collectors are familiar with TSA screening technologies, process and procedures. Data collection includes up to 2 days of burn-in and three (3) to five (5) days of record testing. A test team may consist of three (3) to five (5) Basic Data Collectors and one (1) Intermediate Data Collectors. A test team will travel from one event to another event consecutively for up to 12 events.

A Type 3 OUE event includes one (1) airport test site with one or more screening systems (CAT, alarm resolution, bag search at EDS, passenger arrival, etc.) and multiple screening configurations (Standard

vs. TSA Precheck) during a single event. It is critical that data collectors are familiar with TSA screening technologies, process and procedures. Data collection includes up to 2 days of burn-in and up to three

(3) days of record testing. A test team may consist of one (1) to three (3) Basic Data Collectors and one

(1) Intermediate Data Collector. A test team will travel from one event to another event consecutively for up to 12 events.

Please see Section 8.0 for Modeling and Simulation background information.

2.7 Information Systems Test Support (Tasks A-25, A-26, and A-27)

TSA TED is the DHS designated ITA for TSA’s Financial System Replacement (FSR). The FSR program is a joint DHS initiative to configures a commercial off-the-shelf (COTS) financial system as a replacement for the existing legacy system. The program entered the development and operational testing phase in mid-

2015. Information Systems (IS) testing incorporates an integrated test strategy, which leverages Development Testing (DT) processes through observation and use of test results as part of the operational evaluation of effectiveness and suitability. Typically, integrated testing will include testing associated with the Authority to Operate (ATO) managed by the Information Assurance Division (IAD), 508 compliance testing, continuity of operations testing, user acceptance testing, and cybersecurity testing. Additional testing during operational test includes business operations, interoperability, information assurance, training, reliability, maintainability and availability, and other areas of focus as determined by the test strategy.

IS Test Support includes three (3) types of test events: OT&E, FOT&E, and OUE.

The Government estimates that the following number of IS test events / activities will be required per

Test Execution Event or Data Collection Support Activities

Estimated Number of Events /Activities

Estimated Number of Baseline Days

Estimated Number of Burn-in Days

Estimated Number of Record Days

Information Systems Test Support

IS OT&E 0-1 0 0-2 15-21

IS FOT&E 0-1 0 0-2 8-14

IS OUE 0-1 0 0-2 1-7

There are two (2) general types of test activities expected; development test observation/data collection and operational test data collection. TSA defines developmental test observation as an OUE test event that the ITA is not primarily responsible for planning and executing; rather ITA will participate to provide observation and oversight to ensure the data captured can be used for the overall system evaluation.

Operational test data collection, such as OT&E or FOT&E, is all the data collection activities where the ITA is primarily responsible to plan, execute, analyze and report; the location could be at either a developmental or an operational site.

The Contractor shall manage multiple test events for one or more IT system under test. An OT&E event includes up to two (2) days of burn-in, fifteen (15) to twenty-one (21) days of record data collection. A FOT&E event includes up to two (2) days of burn-in, eight (8) to fourteen (14) days of record data collection. An OUE event includes up to two (2) days of burn-in, one (1) to seven (7) days of record data collection.

Some IT programs have adopted DHS Agile pilot program and, as such, require testing support services for the planning, execution, analysis and reporting for both integrated DT/OT and OT events cycling every eight weeks with up to five separate agile development teams. The scope of each eight-week test cycle is determined by the product backlog at the release planning meeting before each eight-week cycle. The OT test events are two phased; Phase 1 occurs within five business days of the release and is a duration of anywhere between five to ten days. There may or may not be a second phase of OT testing.

This testing is only triggered if a measure is out of threshold during the phase 1 OT and focuses on additional and more rigorous data collection and may last five to ten days. Integrated DT/OT may occur during an eight-week cycle and are one to five days in length. The DHS Agile pilot program tests on business days, within business hours.

An OT&E or a FOT&E test team may consist of up to five (5) Basic Data Collectors, five (5) Intermediate Data Collectors and two (2) Advanced Data Collectors. Additional Advanced Data Collector may be needed to serve as SME.

An OUE of test team may consist of up to five (5) Basic Data Collectors and one (1) Intermediate Data Collector. An Advanced Data Collector may serve as SME on site.

2.8 Cybersecurity Test Support (Tasks A-31, A-32, and A-33)

Depending on the Integrated Test strategy for CPT and CBT operational testing, TSA ITA may conduct cyber resilience testing under this BPA. The ITA typically coordinates with TSA IAD and conducts cyber resilience testing to include, but do not limit to, vulnerability testing, requirements verification, and adversarial assessment. IAD resource or scheduling limitations may result in additional cybersecurity test capacity which may result in this option being exercised.

Cybersecurity Test Support includes three (3) types of test events: OT&E, FOT&E, and OUE.

The Government estimates that the following number of Cybersecurity test events / activities will be required per year:

Support Activities

Estimated Number of Events /Activities

Estimated Number of Baseline Days

Estimated Number of Burn-in Days

Estimated Number of Record Days

Cybersecurity Test Support

IS OT&E 0-1 0 0-2 5-10

IS FOT&E 0-1 0 0-2 5-10

IS OUE 0-1 0 0-2 1-5

There are two (2) general types of cybersecurity test activities expected; test observation/data collection and active test data collection. TSA defines test observation as an OUE test event that the ITA is not primarily responsible for the planning and executing; ITA will participate by collaborating on test planning and provide observation and oversight to ensure the data captured can be used for the overall system evaluation. (See section 3.0 for additional detail)

Active test data collection, such as OT&E or FOT&E, is all the data collection activities where the ITA is primarily responsible to plan, execute, analyze and report cybersecurity test activities. The test environment for cybersecurity assessments could be a developmental / test environment or a production / operational environment.

The Contractor shall manage multiple test events for one or more CPT and CBT system under test. An OT&E or FOT&E event includes up to two (2) days of burn-in, five (5) to ten (10) days of record data collection. An OUE event includes up to two (2) days of burn-in, one (1) to five (5) days of record data collection.

The Contractor shall be prepared to rapidly adjust to and support schedule changes. During testing, the Contractor shall communicate any critical vulnerabilities immediately upon discovery to the stakeholders.

Testing will be paused while stakeholders determine appropriate response. Testing will resume when IS

Security Officer (ISSO) confirmation that testing may resume after disclosure of CRITICAL vulnerabilities.

A test team may consist of up to five (5) Intermediate Data Collector and one (1) Advanced Data Collector for direct observations and data recording. One or more Advanced Data Collector may serve as SME on site and support an eight (8) hour shift daily.

3.0 Test Planning, Analysis and Reporting (PAR) – General

The Contractor shall provide overall coordination and support for operational evaluation, operational test, and other field tests. This includes providing test planning, data analysis, and reporting. These coordination and support activities are referred to collectively as PAR. Generally, PAR will be required for every test event / activity.

For tracking and invoicing purposes, TSA has organized PAR into distinct tasks by the general type of equipment being tested as provided in the chart below:

TASK NUMBER TASK DESCRIPTION

CLIN X001 Checkpoint Operational Test Support A-4 Test Planning, Analysis and Reporting -

Standard PAR Team

A-5 Test Planning, Analysis and Reporting - Additional Support

CLIN X002 Checked Baggage Operational Test Support

A-10 Test Planning, Analysis and Reporting - Standard PAR Team

A-11 Test Planning, Analysis and Reporting - Additional Support

CLIN X003 RCA/ITF Test Support A-16 Test Planning, Analysis and Reporting -

Standard PAR Team A-17 Test Planning, Analysis and Reporting -

Additional Support CLIN X004 RCA Rates and Standards, Modeling and

Simulation Support A-22 Test Planning, Analysis and Reporting -

Standard PAR Team A-23 Test Planning, Analysis and Reporting -

Additional Support CLIN X005 Information Systems Test Support

A-28 Test Planning, Analysis and Reporting - Standard PAR Team

A-29 Test Planning, Analysis and Reporting - Additional Support

CLIN X006 Cybersecurity Test Support A-34 Test Planning, Analysis and Reporting -

Standard PAR Team A-35 Test Planning, Analysis and Reporting -

Additional Support

PAR requires a broad range of scientific and engineering experience to enable the ITA team to validate conformance of capabilities against operational requirements or other test goals. Required Contactor expertise include physics, chemistry, computer science, health physicists, On Screen Alarm Resolution Protocols (OSARP), Human Factors, biometric Identity verification, Statistics, database management, K9 screening, K9 Role Players/Decoys, Behavior Detection screening, Reliability, Maintainability and Availability (RMA), as well as engineering disciplines such as electrical, mechanical, systems, materials, and structural engineering. The Contractor shall provide a mix of system analysts, statisticians, database managers, technical editors and subject matter experts to provide effective PAR services as described in this BPA. The exact expertise required will depend on the type of test and technology being evaluated.

The schedules for operational and field testing are dynamic and are often adjusted rapidly and with short notice to reflect emerging system risks, changing APM’s requirements, test site availability, and performance of the system under test, during both OT and during pre-requisite test phases, such as Qualification Testing (QT). As such, the successful performance of the PAR requirements will require flexibility on the part of the Contactor to adjust to unforeseen situations.

The basic unit of PAR support is a “Standard PAR Team” for a one month for one test event. A Standard PAR Team includes all of the necessary team members (including Subject Matter Experts) to perform PAR…