DEVELOPMENT TEST & EVALUATION SUPPORT SERVICES - TWO (DTSS-2)

PERFORMANCE WORK STATEMENT (PWS)

TITLE OF REQUIREMENT

Development Test and Evaluation Support Services 2 (DTSS-2). All Task Orders under this DTSS-2 will include the language of FAR 52.217-9 “Option to Extend the Term of the Contract” except Task orders issued during the final option period.

REQUIRING ORGANIZATION

Transportation Security Administration (TSA), Acquisition Program Management (APM), Test and Evaluation (T&E) Division.

SCOPE

The Contractor shall provide Development Test and Evaluation Support Services in support of TSA’s responsibilities in all modes of transportation, and in support of stakeholders and organizations associated with transportation security.

The Contractor shall support the planning and execution of technical testing, evaluation, and other data collection activities regarding the extent to which Transportation Security Equipment (TSE) meets TSA’s mission requirement, needs, and specifications.

Additional efforts include test article development and maintenance, validation of training packages, technical evaluation of change quotes, configuration management (CM), standard operating procedure (SOP) development and validation, and hosting technical information exchanges.

Test activities will take place at the TSA Systems Integration Facility (TSIF), manufacturers’ facilities, and other venues as directed by TSA.

BACKGROUND

APM provides security capabilities to enable screening operations to directly support TSA’s mission. APM continually enhances the effectiveness and efficiency of our Nation’s transportation systems through the qualification and deployment of security capabilities that drive the global standard for security. APM’s highly-skilled and diverse team develops and deploys transportation security technologies to detect the range of terrorist and other threats targeting commercial transportation as their primary mission. APM expertise includes applied research, development, operational testing, deployment, and life cycle management of security capabilities to ensure the free movement of people and commerce.

APM develops and procures all technologies for the TSA Checkpoint and Checked Baggage systems at more than 450 Federalized airports in the United States and its territories. APM ’s goals:

• Accelerate the process of delivering new capabilities to the user that improve effectiveness and efficiency.

• Mature business processes that enable APM to make investment decisions that support risk-based security.

• Support risk-driven operations to provide effective and efficient security.

• Work with partners throughout the capability lifecycle to increase mission effectiveness.

The TSA Systems Integration Facility (TSIF) Test Team, consisting of the TSIF Test Branch, Test Infrastructure and Operations Branch (TIOB), and their Contractor support, plans and executes technical testing and other data collection activities in order to provide credible data and information regarding the extent to which TSE meets TSA’s mission requirements and needs to support APM ’s mission and goals. TSA built the 128,000-square-foot TSIF at Reagan National Airport in Arlington, VA. The TSIF enables APM to evaluate transportation security systems, support infrastructure, and mission profiles in a controlled environment.

The TSIF laboratory test areas include:

• Checked baggage screening laboratories consisting of:

o One baggage handling system capable of integrating five checked baggage screening systems o A “stand-alone” checked baggage screening systems area

• Passenger checkpoint screening technology laboratory

• Multi-use area (shared between checkpoint/checked baggage and emerging “Innovation” technologies)

• Explosives Trace Detection (ETD) with small bulk laboratory (i.e., bottle liquid scanners [BLS] and chemical analysis devices)

• Test Article Preparation area hosting a variety of manufacturing equipment and tools

• Fabrication and manufacturing area

• Test tool development area

• Technology Emulation and Simulation Laboratory (TESL)

The TSIF Test Team designs and executes Developmental Tests (DT) and Qualification Tests (QT) to verify whether a product, service, or system complies with regulations, policies, requirements, specifications, and constraints of the TSA operating environment.

The TSIF Test Team supports a wide variety of TSA customers. Efforts range from small scale tests, demonstrations, and tours to elaborate comparative performance tests. Examples of such activities include validation of training packages, SOP development and validation, and hosting technical information exchanges. The comparative performance tests generally involve a statistically sound experimental design to compare baseline processes/systems against alternatives.

TECHNICAL REQUIREMENTS

The Contractor shall adhere to the requirements of following attachments included as part of the Base Blanket Purchase Agreement (BPA), Attachments of this Base BPA, in performance of the requirements of this Base BPA and issued Task Orders.

• TSA Mission; https://www.tsa.gov/about/tsa-mission; The Aviation and Transportation Security Act, passed by the 107th Congress and signed on November 19, 2001, established TSA.

• TSA Strategy; https://www.tsa.gov/sites/default/files/tsa_strategy.pdf

• TSA Administrator’s Intent 2.0; June 2020

• Acquisition Program Management (APM) Strategic Plan; April 2018

• Federal Aviation Administration (FAA) Reauthorization Act of 2018; TITLE VIII—AVIATION REVENUE PROVISIONS; DIVISION K—

TRANSPORTATION SECURITY; TITLE I—TRANSPORTATION SECURITY; Subtitle B—Security Technology; SEC. 1912. TRANSPORTATION SECURITY

ADMINISTRATION SYSTEMS INTEGRATION FACILITY.

• Test and Evaluation Guidebook, Revision 4; December 2019

• Test and Evaluation Policy, Revision 3; August 2018

• Department of Homeland Security (DHS) Management Directive 026-04, Protection of Human Subjects, Revision Number 00; May 2007

• TSIF Test and Test Infrastructure and Operations (TI&O) Integrated Process Manual (IPM) PM4.4-2 (Version 6.0 or current) o If needed move all the PM QMS documents to reference document. See list of docs as of 11.6.20 here (2).

• DHS Directive 026-06, Test and Evaluation, Revision Number 01; May 2017

• DHS Directive Number 102-01, Acquisition Management Directive, Revision 03; July 2015

• TSA Management Directive 300.21, December 2012; February 2019

• Transportation Security Acquisition Reform Act (TSARA), December 2014;(2); GAO January 2019

• TSA T&E, Verification, Validation, and Accreditation (VV&A) Policy, May 2017

• TSA T&E Statistical Methodology Policy Memorandum, April 2016

• DHS T&E Management Guide, Version 3.0; January 2019

• DHS Supplemental Guidance, Threat Assessment in Support of T&E, Version 1.0; September 2018

• DHS Cyber Resilience T&E; T&E Supplemental Guidance; Version 2.0; October 2019

TEST PLANNING, REPORTING, ANALYSIS, AND LABORATORY (INCLUDING OTHER DEVELOPMENTAL TEST VENUE) SUPPORT

APM requires a variety of testing in support of TSA mission requirements. The Contractor shall provide overall management and support for test planning, laboratory staging and logistics, test execution, data reduction and analysis, reporting, and laboratory space restoration. The Contractor shall formulate data collection methodologies and collect the data necessary to complete assigned evaluations, analyses, and support tasks.

Successful performance in this and subsequent sections requires the Contractor to have a diversity of expertise available to support the broad scope of projects addressing past and future mission areas, such as:

• Chemical and biological detection systems.

• Specialized studies such as ETD trace sample efficiency as a function of sampling pressure and sample media.

• Existing TSIF Test Team processes.

• Technicians to help move test articles, act as mock passengers, escort visitors, and collect data.

• Experience as TSA Transportation Security Officers who screen passengers and baggage at airport checkpoints and checked baggage areas.

In the past, the TSIF Test Team conducted test support activities in support of Government agencies internal and external to DHS. TSA participates on a working group that designates lead agencies to support testing for multi-agency use. The TSIF Test Team serves as a primary laboratory test service provider for multiple technologies under this initiative. The Contractor shall provide expert review and support for the development of security technology acquisition documentation. This includes expert review of substantiation provided by technology vendors included in technical data packages related to equipment ability to meet the variety of standards called out in requirements documents, including review of manufacturer provided compliance substantiation (both manufacturer generated and third party generated). For example, the Explosives Detection System (EDS) includes over 500 requirements covering a variety of engineering and scientific disciplines for which the Contractor must substantiate data provided in the Qualification Verification Package (QVP), Qualification Data Package (QDP), and a variety of engineering and developmental test plans to verify. These requirements expand from the base specification to include verification of requirements in referenced documents such as the Checked Baggage Inspection System Interface Requirements Document providing the general, functional, and physical design characteristics of the interface between the EDS and the Baggage Handling System (BHS). Additionally, requirements contained in the Transportation Security Equipment Information Technology Security Requirements document contain both specific requirements as well as reference to additional applicable requirement documents. TSA provides the test documentation formats.

TSA requires employees of all contractors to receive a favorable TSA suitability determination prior to commencing work on the contract.

The Contractors involved in the testing and generation of reports for TSE require access to Sensitive Security Information (SSI). The Contractor shall protect all material marked Sensitive Security Information via TSA approved procedures.

TSA requires a SECRET clearance for all individuals that develop test articles for detection testing, use materials that simulate actual threats, and handle SECRET information. The Technical Monitor (TM) provides input to the Contracting Officer’s Representative (COR) to determine if an individual requires access to classified material and a SECRET clearance. TSA directs the Contractor to initiate a security clearance investigation based on that determination via a formal notification from the COR.

The Contractor shall identify an implementation strategy and perform critical tasks needed to obtain employee security clearances in the most expeditious manner possible during the transition period.

The Contractor shall support base TSIF operational hours described herein Monday through Friday, from 0700 to 1700 as laboratory activities dictate. Occasionally, the Contractor shall support after-hours and weekend work to accommodate schedules as demand dictates. These activities may involve supporting manufacturer activities and continuation of test activities to meet specific timelines. The Contractor shall have steady state, daily resources available at the TSIF to simultaneously continue steady state program management, quality management, planning, infrastructure operations, configuration management, test tools, test articles, test data analysis and test reporting, and other steady-state actions for future/past work and covering laboratory activities while also having resources to perform multiple test execution and data collection activities (link 2) in the TSIF as well as one (1) developmental test activity in an alternate venue. The tests and data collection activities may include any combination of tests occurring in any of the TSIF laboratories.

When demand dictates that execution of simultaneous tests and data collection activities require additional resources not assigned to the TSIF Test Team, the Contractor shall provide surge resources to enable shift work in laboratory spaces to keep up with demand and to allow the addition of technical personnel to conduct work described below. For example, imminent terrorist threats and post terrorist activity require numerous immediate laboratory experimentations simultaneously in addition to the steady state workload, and the Contractor shall provide surge resources as required to conduct these activities. Section 1912 of the FAA Reauthorization Act of 2018 (Pub. L. No. 115-254, Oct. 5, 2018) emphasizes appropriate staffing and resource allocation to prevent unnecessary delays in testing and evaluation of advanced TSE at the TSIF.

In 2018, the TSIF Test Team conducted 451 test execution and data collection activities, 110 tours, and 103 demonstrations while accomplishing all other administrative and logistic activities. The TSIF Test Team serves as a resource for requests for support from numerous groups both in and out of TSA on a daily basis. Individuals within TSA organizations can request laboratory support through use of an event request form.

The Contractor shall provide TSIF Testing Support Requirements as follows:

Program Management A. Provide timely and accurate day-to-day assignment of personnel to functions or actions in this BPA PWS to manage workload and test planning, execution, and reporting due dates, map personnel actions to PWS requirements to COR, TM, and TSIF Test Team leadership by Thursday before the next week (or have information available real time on a dashboard located on the network) 99% of time.

B. Participate in the daily TSIF Test and Evaluation Operations Morning Meeting to ensure staffing for laboratory activities, maintain the activity calendar, and participate in weekly staff meetings covering updates to all established laboratory projects.

C. Attend TSA program offices’ Integrated Product Team (IPT) and T&E IPT meetings, T&E Division Evaluators’ System Evaluation Team (SET) meetings, and internal TSIF Test Team meetings as directed.

Quality Management

D. Support TSIF Test Team’s SOP and work instruction development, refinement, training, and audit per Integrated Process Manual (link). TSIF Test Team’s has over thirty-five (35) documented processes and work instructions.

E. Maintain the accuracy and appropriateness of the TSIF Test Team’s Quality Management System (QMS) documentation, including Quality Policy, Quality Manual, and keep it current by:

o Administering TSIF’s independent ISO certification including arranging for certification audits including hosting the auditor during the entire audit o Supporting entrance and exit management briefings o Arranging for personnel to participate in the audit o Following up on actions to address findings o Administering the TSIF Test Team’s On-the-job training to include any Government administered training o Implementing the TSIF Test Team’s continuous improvement, corrective action, and customer satisfaction and feedback processes.

F. Per Test Tool Process version 3.1, provide expert support, including process audit conduct, corrective action tracking, performance metric development, and coordination with the International Organization for Standardization (ISO) / International Electro-Technical Commission (IEC) registrar to sustain and expand the scope of TSIF Test Team’s ISO 9001: 2015 Certification covering over thirty-five (35) documented processes and work instructions and gain ISO/IEC 17025 Certification. The TSIF Test Team continuously adds and refines work instructions and processes as part of the TSIF Test Team’s Quality Management System (QMS). QMS covers CM as a topic. TSIF complies with ISO 17025 for specific chemistry related processes and test tool development and fabrication (e.g., code development, robotic design, and fabrication).

G. Provide expert support to establish TSIF Test Team compliance with the requirements of ISO/IEC 9001: 2015 and 17025 (General Requirements for the Competence of Testing and Calibration Laboratories) to the existing program or apply to additional spaces of the TSIF’s lab or scope and application. This support includes process development/modification, internal audit conduct, corrective action tracking, and performance metric development. The Contractor shall coordinate with the National Institute of Standards and Technology (NIST) guidance, knowledge transfer, and consultation to the TSIF Test Team including mock internal and external audits. The Contractor shall identify and establish a relationship with a private-sector accreditor if TSA directs at such a time where the TSIF Test Team demonstrates maturity and compliance to the standard.

H. Control and upload 99% of TSIF Test Team documentation on a TSA-provided file storage system. Currently, the TSIF Test Team relies on iShare and networked storage for maintaining TSA ISO/IEC QMS related documentation and automated customer satisfaction queries to test sponsors. The Contractor shall coordinate with TSA organizations responsible for maintaining these systems to arrange for corrective actions and to expand TSA’s utilization of these systems.

I. Maintain compliance with laboratory protocols for human subjects and participate in Institutional Review Board (IRB) application and information processes. Perform administrative functions to maintain relationship with TSIF Test Team’s IRB and ensure compliance of the TSIF Test Team and TSIF’s labs with IRB requirements (e.g., including screening passengers as well as passenger luggage). Administrative functions include financial management, keeping IRB required statistics concerning participation, coordinating approvals for protocol changes and survey questions, and responding to IRB queries.

Planning J. Support IPT functions including documentation development, review, comment, constructive edits, and recommendations to documents such as Test and Evaluation Master Plan/Qualification Master Plan development, System Evaluation Plan reviews, Qualification Verification Package (QVP), Qualification Data Package (QDP) reviews (and other Technical Data Packages), vendor test documentation, and other IPT related documents within due dates. These documents may include information related to throughput, safety (Safety Review Board Template [PM2-T07]), reliability, maintainability, and availability (RMA), physical characteristics (floor loading for example), fault recovery, maintenance, interfaces (baggage handling system [BHS] and communications, network [both system multiplex and Security Technology Integrated Program {STIP} communication requirements]), human system integration, status and archive, cybersecurity, fault generation, display quality, power, environmental (operational environment), and regulatory requirements. The contractor will staff technical data package reviews with an appropriate mix of expertise to cover all required subject matter and all suites of equipment (i.e., body scanner, ID scanner, carry-on baggage, checked baggage, trace, bottle liquid, or any additional TSA product lines).

K. Produce test plans and reports for large and small scale tests and data collection activities in all TSIF laboratories (see plan and report templates on the TSIF Test Team’s iShare page). The Contractor shall use the TSIF Qualification Test (QT) Plan and QT Report templates for both Qualification Tests and Regression QTs at the TSIF. Regression QT are smaller in scope compared with full QTs. Regression QTs only include the necessary sections relevant to the scope of the Regression QT and the Change Request. The Contractor shall use the TSIF Data Collection Activity Plan (DCAP) and Data Collection Activity Report (DCAR) for developmental TSE and non-acquisition TSE activities at the TSIF that do not support a formal TSA acquisition decision. The TSIF Data Collection Activity Plans (DCAP) are not sufficient for Regression QTs. See TSA’s T&E Policy Appendix B: Document Guidelines and TSA’s T&E Guidebook for approximate due dates. The Contractor shall negotiate due dates with the respective Federal TSIF Test Director. The Contractor shall technically edit each deliverable per TSIF Document Review Process 7.5.2-1.

L. Utilize TSA’s Statistical Methodology Policy and DHS T&E Management Guide Appendix C. DHS Scientific Test and Analysis Techniques when developing statistical design of experiments (DoE) and analyzing data for test execution and data collection activities and document them in the respective plan and report to include statistical confidence point estimates and confidence intervals. Explain how DoE facilitates the design, and include the goals, metrics, factors, and levels that impact the outcome of the test. Include a strategic method for varying those factors and levels across TSA designated critical scenarios (i.e., detection), and appropriate statistical power and confidence levels for important responses for which they make sense. The DoE explains how the Contractor shall control and record the conditions to understand their effect on the TSE performance. The TSIF Test Team uses the DoE for test execution and data collection activities. The Contractor shall investigate opportunities to utilize statistical methods such as hypothesis testing across all projects where appropriate to do so and reviewing test results for adequacy.

M. Observe testing at external test facilities to include manufacturer locations, third party testing organizations, or any other external test facility and provide an observation report per T&E Guidebook Appendix (Data Sources). Add technical review of external test plans and reports. For planning purposes, approximately six (6) observations of manufacturer’s internal tests per year with one (1) week duration for test observation.

Test Execution/Data Collection N. Support small scale laboratory activities documented through a TSIF ERF submittal (Test Event Detail Checklist [PM2-T01], Completed TSIF

Event Detail Form [PM2-T03]), in addition to the test and tour activities. The Contractor shall support small scale laboratory data collection activities at a rate of approximately two (2) per day (not including tour related ERFs described separately herein) averaging thirty (30) hours per week with required support of two (2) to three (3) Contractor FTEs per week. While small scale laboratory data collection activities average thirty (30) hours per week, some may require only fifteen (15) hours per week, while others require forty (40) hours. Small scale laboratory support includes the development of TSIF data collection activity plans and data collection activity reports, equipment demonstrations, equipment exercises, equipment training, “quick see” system response tests, validation of field SOP, and training product development support. Another example includes the Contractor designing and setting up robotic systems to conduct reliability and safety radiation assessments, which require minimal monitoring and adjustments during the execution.

O. Support simultaneous execution of all test and data collection activities documented through a TSIF Event Request Form (ERF) submittal that contains the who, what, where, when, and why information needed to plan and execute the activities. The Contractor shall support simultaneous execution during periods where workload exceeds the full scale staff through the use of surge resources after approval from the Technical Monitor and/or Contracting Officer Representative. The Contractor shall support full scale test execution and data collection activities with required support of at least thirty (30) Contractor Full Time Employees (FTE) per week. While full scale test execution and data collection activities average forty (40) hours per week, some weeks may require only twenty (20) hours, while others may require two

(2) hours. Full scale test support includes the development of test plans and test reports.

P. Operate TSE at the TSIF per TSA TSO training and all user access levels of TSE (i.e., levels z, 1, 2, and 3). Contractor shall explain TSE capabilities, limitations, and design for all tours and demonstrations at the TSIF. Contractor shall oversee and document equipment maintenance and system upgrades. TSIF houses over thirty (30) unique pieces of TSE.

Q. Perform safety assessments in accordance with TSIF Test Team’s QMS Training and Process. Contractor shall develop safety mitigation strategies for any OSHE or TSIF Test Team identified limitations related to safety and present them to a safety board prior to commencement of test activities.

Infrastructure and Operations R. Provide laboratory oversight and technical support to operate transportation security equipment and data collection equipment housed in laboratory facilities.

S. Provide support for scheduled and unscheduled TSIF tours and system demonstrations as directed. Historically TSIF Test Team hosts approximately seven (7) to twenty (20) tours and demonstrations per month.

T. For routine and preventative maintenance of TSIF equipment, establish required services, then contact service providers, track accomplishments, and coordinate other maintenance activities as test operations need in a timely and accurate manner 98% of the time without coordination issues. Coordination includes scheduling, arranging access, ensuring approved configurations changes, and ensuring adherence to TSIF Test Team’s SOPs including ensuring that they escort personnel (as required) and that the Contractor staff does not conduct work outside the approved scope per TSIF Test & TIO Integrated Process Manual, PM8-F02: TSIF Security Equipment Maintenance Checklist, and PM8-P03 TSE Logbook OI.

U. Provide logistic support for test participants, including: developing criteria for selecting participants, disseminating requirements for arrivals/departures and lodging locations, providing participants with information on parking/transportation options, verifying facility access, in-briefing, designing/directing participant movements through test stations/scenarios, ensuring food availability and directing breaks, collecting data from participants, and providing other similar participant logistic support as test needs dictate. For Transportation Security Officers (TSOs) supporting test activities, provide: administrative, coordination, and logistic support for the duration of their activities at the TSIF. Provide logistics support for test and data collection activities (i.e., staging bags, data collection equipment, etc.). See TSIF Test & TIO Integrated Process Manual, TSIF IRB Compliance Process (PM0-P08), TSO Request Process (PM3-P01), and Mock Passenger Support Process (PM3-P04).

Configuration Management V. Manage CM activities at the TSIF including physical configuration audits of security technologies (TSA owned or bailed to TSA, systems entering the TSIF both new entry and legacy systems). Coordinate review and/or implementation of configuration changes quotes resulting from Developmental Deviations (DEP) and the Engineering Review Board (ERB) process. Participate and coordinate review and/or implementation of configuration changes resulting from the TSA Configuration Control Board (CCB) and Developmental Configuration Control Board (DCCB) process. The TSIF Test Team processes an average volume of fifteen (15) change packages per month with a range from six (6) to twenty-six (26) change packages monthly. These change packages average approximately twenty (20) pages in length, but can include up to hundreds of pages. Contractor shall adhere to 2600.4 PM0-P14 TSIF TSE Configuration Management Plan Version 2.1.

W. Lead efforts associated with establishing initial as-tested developmental configuration baselines for systems entering the TSIF, including assembling and verifying configuration item documentation for candidate procurement and developmental systems under test to the appropriate level of detail to document the performance, functional, and physical attributes commensurate with the complexity of both the systems and requirements. Conduct a critical functions analysis and gap analysis to ensure that the quoted Configuration Items (CIs) reflect those critical elements (hardware, software, and firmware associated with the complex equipment) to the system performance against the technical, functional, and operational requirements the TSIF Test Team evaluates. The configuration management process also includes test articles (i.e., bag sets and threat targets; Operating Instruction PM0-P02 – v2.0 – FINAL TSIF Test Tools and Test Articles Configuration Management), specially designed test instrumentation, and support equipment. Review technical documentation and participate in discussions with various stakeholders and the equipment manufacturers to understand the functional and physical characteristics of systems and to advise the Federal Test Directors on those items that TSA designates as CIs.

Tools

X. Establish information backup alternatives as TSA directs to mitigate instances of limited system functionality and loss of data maintained on TSA controlled networks including exploitation of Test and Evaluation Monitoring Network (TEMNET), local storage capability, and manual processing of customer satisfaction surveys. This system hosts test data (including video) and the TSIF Test Team utilizes the platform to develop test tools to increase automation and efficiency for the TSIF Test Team. Within 40 business days of starting period of performance, the Contractor shall perform weekly backups and monthly offsite backups.

Y. Design, develop, fabricate, and manufacture software and hardware test tools to improve test efficiency and accuracy, and expand the scope of the TSIF Test Team’s capabilities like data collection, analysis, and task automation. The Contractor shall process a Test Tool request and have a deliverable and development strategy agreed to with the relevant TSIF Test Director and TM per test tool project per P10-P01 within 1 week of Test Tool request. Development efforts include establishment of documentation including system requirements, verification/validation plans, validation/verification reports, test tools/test articles configuration management (Operating Instruction PM0- P02 – v2.0 – FINAL TSIF Test Tools and Test Articles Configuration Management), integration into TEMNET environment, and other supporting documentation as appropriate for the specific tool application. The Contractor shall maintain and update test tools in accordance with TSIF Test Team’s SOPs. TSA or Contractor personnel recommendations with TSA approval initiates the Contractor’s tool development efforts. Examples of past test tool development efforts include robotics, automated passenger positioning verification (Advanced Imaging Technology (AIT) Pose Tool), AIT Response Evaluation Tool (ARET), Checked Baggage Resolution Area Timing Application (CTA), Automated Monitor Tool (AMT).

Z. Provide expert users of the TEMNET hosted in all lab spaces. TEMNET provides data management, video analytic capability, and database and test tool hosting to support the TSIF Test Team. Support further refinement and expansion efforts to increase reliability and test team utilization of the system, including use of the system for vital test information back-up.

AA. Operate, maintain, and train others on the use of all laboratory data collection equipment per user manual and training. Laboratory data collection equipment includes diagnostic equipment, measurement equipment, manufacturing tools, material logistic support equipment, test instrumentation, computer aided design (CAD) software, modeling and simulation tools, high-performance liquid chromatography (HPLC), liquid quality control, TekScan IScan System, function generators, oscilloscopes, microscopes, meters of all types, computer aided design and manufacturing equipment including computer-aided design (CAD) and three dimensional (3D) printers, gas chromatograph (GC), laser etchers, version control systems such as Git and Subversion, C#.NET desktop and ASP.NET web applications, and other laboratory tools required to support the TSIF Test Team.

BB. Integrate test activity data into electronic tools and databases and provide all collected test activity data in a validated electronic format unless the TSA Technical Monitor and/or COR directs otherwise.

CC. Support the TSIF Test Team with all appropriate disciplines to cover the requirements including cybersecurity compliance and adversarial assessments as an emphasis with evolving DHS requirements. The tests identified herein may include the following: code analysis through system-level testing, vulnerability assessments of the system and interfaces, and probe testing. Contractor shall verify system compliance with cybersecurity compliance and adversarial assessments that may include security compliance tools such as Nessus, Retina, nmap, and Wireshark; code analysis through system-level testing; vulnerability assessments of the system and interfaces; and support for penetration testing based on cyber threat assessments. The Contractor shall liaison with TSA Information Technology (Information Assurance and Cyber

Security Division) for TSA IT’s Cyber Resilience Testing by providing Cyber Resilience SMEs to provide in-house expertise and support to TSA IT, execute cyber resilience testing for TSA IT with TSA IT input, or work with Information System Security Officer (ISSO) assigned to the TSIF.

DD. Provide knowledgeable users of the systems and capabilities hosted in the TESL and support TESL capability expansion. Contractor shall ensure these systems have sufficient equipment that includes a variety of EDS simulators and emulators as well as equipment associated with the Common Element Architecture (CEA), Common Graphical User Interface (CGUI) Beta, Field Data Reporting System (FDRS), and Universal Simulation Platform. Support expansion of the capabilities to allow for image download and storage, image uploads, emulators, simulators, image quality evaluation tools, and human factors tools. Utilize the capabilities to perform statistical evaluations of the contents of images obtained from fielded systems to assist in maintaining the contents of the “stream of commerce”. Support TSIF Test Team migration of test tool development capabilities to a different location at the TSIF.

EE. Perform testing to establish and assess received documentation associated with Verification, Validation, and Accreditation of simulants, stimulants, test articles, and team-developed data collection and test equipment. Additionally, verify and validate simulators (human interface identical) and emulators (system algorithm identical) that the manufacturer delivers to replicate the respective detection system performance for use as training devices and to allow system detection algorithms to run on image files.

Reporting FF. As needed, develop a Quick Look Report (QLR) in MS Excel or MS PowerPoint format providing the initial test findings in addition to a final report. TSA usually requires the Contractor to provide the QLR within fifteen (15) business days after the final Data Authentication Group (DAG) meeting. On occasion, TSA may request the Contractor provide expedited QLRs a few business days after test execution based on the scope of testing (volume of information to report), as well as provide an interim issues report at a designated time during test execution.

TSA utilizes the QLR as a basis to adjudicate (or score) results with the operational proponent.

GG. Provide a Weekly Status Report of all TSIF Test Team projects and activities. This report provides a status of all testing across all platforms, TSIF infrastructure lab operations summary, QMS and ISO processes and audit status, TSIF Test Article status, and Data Collection Tool Development to include the next action for each (who, what, and when). The Weekly Status Report details the technical progress of work activities, future scheduled project and task activities, and any associated logistics efforts managed under the respective Task Order.

HH. Provide a written Monthly Status Report that highlights major accomplishments, and all issues (TSA and Contractor identified issues related to DTSS-2 requirements and other challenges the TSIF Test Team needs to mitigate) in each functional area/respective Task Order that need resolution/decisions/actions (who, what, and when), and a summary of all active and closed Task Orders. The summary of the Task Orders includes obligations/expenditures to date, award value, summary of awarded modifications, status of Key Personnel, and Period of Performance. This report summarizes the Financials of the Task Order(s) to include funds obligated to date per CLIN, test execution and data collection activities completed, pending and paid invoices submitted, and associated Other Direct Costs (ODCs) (e.g., travel, laboratory equipment, and shipping) that test execution and data collection activities incur so that TSA understands the total price/cost of the given test event and DTSS-2 requirements as whole. The Contractor shall submit this report within ten (10) business days following the end of the previous month.

II. Provide and conduct a Quarterly Program Management Review (PMR) at a location negotiated with TSA. At the PMRs, the Contractor shall present their progress in supporting the respective Task Order, status on all technical requirements in this Performance Work Statement (PWS), identify any issues requiring TSA intervention, action, and response (TSA and Contractor identified issues related to DTSS-2 requirements and other challenges the TSIF Test Team needs to mitigate, who, what, and when), identify quoted activities for the next quarter, and data including, but not limited to scope, financials to date, and schedule. The Contractor shall provide PMR minutes, including a summary of action items and highlighting any issues requiring resolution within one (1) week following the conclusion of the PMR.

TEST EXECUTION / DATA COLLECTION ACTIVITIES

The Contractor shall price all test planning, data reduction and analysis, and reporting activities under “Test Planning, Analysis, and Reporting”.

While TSA’s Request for Quote describes the annual estimates of required test execution, the actual number of test execution and data collection activities at the TSIF required and ordered may end up less than or greater than the estimated quantities provided. TSA provides these estimates so that the Contractor may price the supporting “Test Planning, Analysis, and Reporting” requirements associated with the estimated test execution and data collection activities.

A fair amount of vendor design and experimentation takes place during the system installation; therefore, the Contractor can expect to host vendors and collect observation data during the initial system installation and vendor self-testing phase. Once TSA establishes a system configuration in accordance with CM requirements in this PWS, frequent system upgrades also take place which may require cursory regression testing and CM audits.

The Contractor shall price for unexpected system reliability and other factors that can temporarily suspend testing and data collection activities.

System reliability and logistics downtime can temporarily delay testing for up to one (1) month depending on the nature of the failure.

The Contractor shall support test execution and data collection activities in support of the TSA Checkpoint Technologies Division, Checked Baggage Technologies Division, Requirements and Capabilities Analysis, and other DHS and TSA sponsors. The Contractor shall support studies to validate modeling and simulation results and predict system response to new technology immersions, equipment configuration, and protocol changes.

The Contractor shall provide “mock passengers” for a pre-defined duration, dependent on requirements per passenger screening activity at the TSIF, to support studies such as comparisons of TSE system effectiveness. With the exception of Qualification Testing and the portion of testing requiring mock passengers described herein, general passenger screening and regression test activities may have advance notice prior to test execution of as little as three (3) business days prior to the expected execution. This depends upon whether the Contractor can exploit previous test plans and the scope of the activities allows the Contractor to generate a plan within this timeframe. The Contractor shall make internally sourced Mock Passengers available and ready to start within one (1) week of notice and make externally sourced Mock Passengers available and ready to start within two (2) weeks of notice. While TSA does not require Mock Passengers to undergo TSA’s full Security Suitability Determination, they do undergo an expedited cursory background check, usually completed within 24 – 48 hours from Contractor submission. The Contractor shall assess the individuals’ ability to satisfy a successful TSA cursory background check and must only submit individuals that can meet TSA’s general security requirements.

The Contractor shall test integrated checked baggage systems utilizing the TSIF baggage handling system, as well as test units in stand-alone configurations for these test execution and data collection activities.

In addition to formal test execution and data collection activities in support of formal acquisition test requirements, the Contractor shall support test execution and data collection for developmental products supporting a variety of test and technical services associated with introducing new systems to the screening environment. These developmental systems may not have established requirements and therefore also may require TSIF technical personnel to collaborate with other TSA representatives to review quoted test scope documentation to ensure testable and measurable “requirements”.

These test execution and data collection activities support acquisition decisions and support such activities as system upgrades (Request for Deviation evaluations conducted prior to operational fielding scaled in accordance with the significance of the quoted change), testing associated with issues communicated from the operational environment, and equipment operation for TSA’s training curriculum development. Test execution and data collection can also involve On-Screen Resolution (OSR) performance, risk-based screening activities, Checked Baggage Resolution Area (CBRA) performance, and system effectiveness assessments for specific test articles of interest.

Test execution and data collection activities also routinely have burn-in phases to ensure establishment of proficient test personnel users and provide stakeholders an understanding of system operation, support establishment of SOPs/Concept of Operations (CONOPs), allow for the conduct of TSA Office of Safety, Health, and Environment (OSHE) reviews to ensure passenger and system operators safety, and TSA Office of Information Technology (OIT) personnel to conduct system scans.

Some TSE require repeated regression tests as TSA and vendors modify system configurations to address findings from previous tests and to implement TSA requested enhancements.

Test execution and data collection activities may include, but are not limited to, body scanners, metal detectors, checkpoint X-ray, checked baggage X-ray (stand alone and integrated), baggage handling systems, simulators, emulators, authentication technologies (including biometrics), board pass scanners, explosive trace detectors, bottled liquid scanners, behavior detection technologies, chemical and biological detection systems, Security Technology Integrated Program integration utilizing a TSA owned STIP Enterprise Mission Manager (STEMM) platform, and cargo, intermodal, and multipurpose technologies.

The Contractor may only invoice and receive payment for successfully executed/completed test execution and data collection activities. For example, if TSA ordered resources for two (2) weeks of TSA funded testing, but the testing only ends up taking one (1) week, then the Contractor only invoices and receives payment in support of one (1) week.

TEST ARTICLE DEVELOPMENT, MAINTENANCE, AND SUPPORT

Per PM9-P01 Test Article Program version 5.0, the Contractor shall develop, procure, test, validate, manufacture, control, and maintain TSE Test Articles and Test Kits required for testing in the field, prohibited item detection testing for TSA certification of detection algorithms, Security Equipment Performance Tests (SEPTs), and other TSA T&E activities as directed. Test articles include threat simulants, inert improvised explosive devices (IEDs), 3D printed weapons, firearms, sharp objects, blunt objects, grenades, and artfully concealed threat items to test TSE. The Contractor shall prepare a specification and associated drawings for each type of test article and test kit developed. The Contractor shall deliver and maintain

(1) test article inventory tools, (2) test article specifications and drawings for developed TSE Test Articles and Test Kits, (3) any software developed to analyze TSE Test Article and Test Kit produced images to include documentation, and source and object code, (4) any databases developed to track security equipment performance based on results obtained from utilizing the TSE Test articles and Test Kits, (5) any operator manuals required to utilize the TSE Test Articles and Test Kits, and (6) any tools developed to generate the Test Articles, to include source code, executable documentation for using such tools, documentation of the installation environment, and execution prerequisites.

The TSA provides all space, tooling, and raw material storage for manufacturing/validating required test articles (non-live).

The TSIF Test Team has established multiple bag sets, each consisting of 500 “stream of commerce” articles of checked or carry-on luggage that the Contractor must maintain statistically equivalent to general passenger and checked baggage luggage content of the traveling public. The Contractor shall obtain and process luggage image data from various locations to maintain the luggage sets containers and contents. The Contractor shall refresh the bag sets at least once a year or as TSA dictates. TSA may also require that the TSIF Test Team and Contractor address passenger behavior changes (due to increased fees in checked luggage affecting profile of carry-on luggage for example).

The majority of tests conducted utilize existing test articles with slight modifications/augmentation depending on the experimental design from the TSIF Test Team (Statistics, Design of Experiments Subject Matter Expert). The Contractor shall purchase and construct “targets” used within the existing test article sets for the specific test execution or data collection activity.

A. Checkpoint Test Articles

The Contractor shall provide TSE test article support for Checkpoint TSE that includes the Credential Authentication Technology (CAT), Boarding Pass Scanners (BPS), ETD, Advanced Technology (AT) X-ray, checkpoint Computed Tomography (CT) scanner, AIT, BLS, and others.

The vendor and equipment models depend on test requests from TSA program managers and other customers.

The Contractor shall acquire and maintain a sufficient quantity of TSE test articles for each Checkpoint TSE technology to meet scheduled T&E activity requirements. For TSIF laboratory activities, the Contractor shall acquire and maintain test articles sufficient to satisfy the statistical rigor required to satisfy the test design. CAT and BPS Test Article Kits contain a variety of industry airline Boarding Passes (BPs), updated on an as-needed basis not to exceed twenty-five (25) BPs per kit, and identification documents (IDs) representative of those issued to the traveling public, updated on an as-needed basis not to exceed twenty-five (25) IDs per kit. In addition to baggage and SOC items, the Contractor also needs to obtain and maintain both liquid explosives Standards as well as Thumbprint Quality Control kids. ETD and AT Test Article Kits contain approximately fifteen (15) representative stream-of-commerce checkpoint luggage physical bags and representative stream-of-commerce Checkpoint luggage digital bag datasets. BLS Test Article Kits consist of twenty-five (25) bottle containers of varying types containing a variety of liquids and stimulants identified after award. The Contractor shall develop AIT Test Article Kits as TSA directs.

Iterative validation of test article sets utilizing TSE and configuration capture activities also take several business days to complete.

B. Checked Baggage Test Articles

The Contractor shall provide TSE test articles to support Checked Baggage TSE. The vendor and equipment models depend on test requests from TSA program managers and other customers.

The Contractor shall acquire and maintain a sufficient quantity of TSE test articles for each Checked Baggage TSE technology to meet scheduled T&E activity requirements. For TSIF laboratory test activities, the Contractor shall acquire and maintain test articles sufficient to satisfy the statistical rigor required to satisfy the test design. For operational test support, test article sets generally are between fifteen (15) to twenty-five (25) bags with appropriate “clutter” and artfully concealed simulant IEDs. Iterative validation of test article sets utilizing TSE and configuration capture activities also take several business days to complete.

C. Air Cargo Test Articles

In support of the Cargo, Intermodal, and Multipurpose test execution and data collection activities, TSA requires the Contractor to furnish test articles supporting the specific event.