Annex_TTA.Descriptions.v1_FINAL.docx

ANNEX: W900KK-22-R-0012

SPECTRUM SYSTEMS TEST (SET)

The T&E/S&T Program is charged to enable development of technologies required to test and evaluate future warfighting capabilities. To do this the Program uses multiple Test Technology Areas (TTAs), where each either focuses on emerging warfighting capabilities within a specified technology genre (as discussed later in this document) that present T&E challenges, or focuses on a broader infrastructure challenge (such as, significantly improving range instrumentation or transforming test range data communications). Each TTA is executed by a team led by an Executing Agent (EA), who is a Government employee that typically works at a Service T&E field organization. While the EA works for a particular Service, the TTA addresses test technology needs spanning DoD-wide, across all three Services (Army, Navy, and Air Force) as well as other DoD organizations (such as the Missile Defense Agency).

A prime responsibility of the EA is to lead a multi-Service, subject matter expert working group, consisting of members of the DoD S&T and T&E communities, to formulate and/or validate the topics of interest for the BAA issued for the TTA. Each TTA manages a portfolio of technology development projects.

Details about each TTA’s charter, sub-categorized technology domain descriptions, and test technology shortfall focus areas are listed in later sections within this document per hyperlinks below:

· Autonomy and Artificial Intelligence Test (AAIT)

· Advanced Instrumentation Systems Technology (AIST)

· Command, Control, Communications, Computers, Intelligence, and Software Intensive Systems Test (C4T)

· Cyberspace Test Technology (CTT)

· Directed Energy Test (DET)

· Electronic Warfare Test (EWT)

· High Speed Systems Test (HSST)

· Space Systems Test (SST)

· Spectrum Efficient Technology (SET)

ANNEX: W900KK-22-R-0012

ANNEX: TRMC T&E/S&T PROGRAM TEST TECHNOLOGY AREA DESCRIPTIONS

A-1 (Return to Cover Page) Autonomy and Artificial Intelligence Test (AAIT) led by U.S. Naval Air Systems Command (NAVAIR), Patuxent River, MD. Formerly known as Unmanned and Autonomous Systems Test (UAST) in prior BAA releases, AAIT develops technologies that significantly advance the science of testing unmanned and autonomous systems (UASs), which encompasses artificial intelligence and machine learning, to improve the safety and user trust in UAS tests and operations. Technologies developed under this TTA include test planning for UASs, test execution and control of UASs, and assessment of UAS performance. The goal of this TTA is to develop technologies that effectively address the challenges specific to testing Artificial Intelligence-based and autonomous systems for operations across the full spectrum of battlespace environments. The overarching goal of AAIT is to increase user trust in AI-based and autonomous systems.

Within the T&E/S&T Program, AAIT T&E needs are grouped into six domains:

Autonomous Systems Test Planning, which includes technologies to design autonomous system test plans in a timely manner, predict autonomous behavior for testing, and assure thorough testing of autonomous systems;

Autonomous System Test Execution and Control, which includes technologies to enhance safety of autonomous system testing; create simulated test environments that are complex, immersive, and reactive; and adapt a test environment to a cognitive autonomous system;

Autonomous System Performance Assessment, which includes technologies to assess performance of autonomy’s functional components, measure human-autonomous system interaction, measure autonomous system reliability and performance, and assess vulnerabilities of autonomous systems;

Autonomous System Orientation and Location, which includes the ability to determine precisely where UAS are on the ranges and how they are oriented to ensure position or orientation doesn’t violate range safety limits. If errors are found, how can errors be remediated. Also included is the ability to verify that the UAS can locate elements in the environment correctly, correct any errors, and generate synthetic logs to assist in safety case analysis, and lastly the ability to compare the UAS location, as perceived by itself, to the UAS location as determined by external range sensors. Technology in this domain will improve safety while on live test ranges. Assess performance of UAS sensor systems;

Autonomous System Machine Vision, which includes the ability to evaluate the realism of synthetic images in terms of their utility for testing machine visions systems, the ability to assess if images, synthetic or real, have the features required to test or train a given machine vision system, the ability to assess vulnerability of blue AI-based machine vision systems to adversarial imagery, and lastly the ability to assess vulnerability of the learning aspects of blue AI-based machine vision systems to corruption by adversarial imagery; and Counter Unmanned Aerial Systems (C-UAS), which includes technologies to assess performance of a blue C-UAS in modeled environments to predict behavior in advance of live flight test, and to build trust in the use of blue C-UAS.

Major test technology needs include the following:

· Ability to assess robustness of UAS software

· Ability to define metrics to prove a holistic assurance case for UAS

· Ability to assess human-machine effectiveness

· Ability to rapidly plan and execute an effective and efficient strategy for the T&E of UASs

· Ability to assess the performance of UASs interacting in their operational environment, including the ability to test their awareness of and response to geospatial constraints

· Ability to create a virtual proving ground by capturing and organizing large amounts of real data in order to embed a UAS under test in an artificial environment that is “sensor realistic” with significant (at least 10x) reduction in cost to develop virtual sites

· Ability to assess the autonomous behavior of UASs, especially unmanned aerial vehicles, while maintaining a safety override (i.e., ability to terminate a test if rules are broken) at all times with minimal modification and impact to a UAS under test

· Ability to understand vulnerabilities of UAS

ANNEX: W900KK-22-R-0012

AUTONOMY AND ARTIFICIAL INTELLIGENCE TEST (AAIT)

The U.S. Naval Undersea Warfare Center, Newport, RI, leads the Advanced Instrumentation Systems Test (AIST) Technology Area. AIST develops advanced instrumentation, sensors, data storage, and power sources to support continuous, non-intrusive testing. The goal of this TTA is to develop and demonstrate critical enabling technologies in support of data acquisition capability that includes Time Space Position and Information (TSPI) sensors, advanced sensors focused on miniaturization, low-weight, self-powered, and interoperable components able to be installed in any system under test in a low-cost, unobtrusive manner. This TTA advances the field of device physics by investigating innovative materials, micro-electro-mechanical systems (MEMS), miniaturized power and energy sources, data transformation, and novel packaging technologies to support the T&E of warfighting systems.

Within the T&E/S&T Program, AIST T&E needs are grouped into six domains:

Advanced Sensors, which includes sensors for non-intrusive evaluation of system performance in harsh environments with miniaturized instrumentation Advanced TSPI, which includes high-precision TSPI and GPS-denied TSPI especially for individual warfighters, highly dynamic guided weapons, undersea weapons systems, and hypersonic weapon applications Advanced Energy and Power for Instrumentation, which include warfighter and weapon systems as well as systems required for test instrumentation Microelectronics, which includes decreased size, weight and power (SWaP) of instrumentation, advanced metadata concepts, adaptive computing data compression, onboard data transport connectivity media, and microminiaturization of electronic components Human Performance Measurement and Assessment, which includes assessing individual and teams of warfighter mental and physical performance and interactions between the warfighter and unmanned systems Range Environmental Encroachment, which includes mitigating the effects that air, sea, and land ranges have on the environment including wildlife and encroachment by public systems on ranges Major test technology needs include the following:

· Ability to gather high-precision TSPI, when GPS signals are blocked or unavailable and GPS receivers in ultra-high dynamic environments

· Advanced power and energy sources including miniature fuel cells, super capacitors, energy harvesting devices, and at-sea remote battery charging

· Advance sensors for personal armor evaluation, energetics, optical tracking and camera systems, data reduction, precise and accurate instrumentation using radar technology, warfighter/weapons/sensors position and orientation, and munition/warhead characterization testing in extreme environments

· Ability to test humans as systems, to include human-machine interfaces and improved methods to measure warfighter cognitive and physical workloads as well as human and unmanned device interactions

· Ability to mitigate the impact on T&E due to conventional energy sources (e.g., power lines) and alternate energy sources (e.g., wind turbines)

· Ability to mitigate open-air and undersea testing constraints generated by endangered species habitat and protected marine mammals

· Ability to test & measure real-time casualty assessment (RTCA) and counter-small unmanned aircraft system (C-sUAS) system performance

ANNEX: W900KK-22-R-0012

ADVANCED INSTRUMENTATION SYSTEMS TEST (AIST)

C4I and Software Intensive Systems Test (C4T led by the U.S. Naval Air Warfare Center Weapons Division (NAWCWD), Point Mugu, CA. Formerly known as Net-Centric Systems Test (NST) in prior BAA releases, C4T develops models, simulations, software, and devices to measure and assess the performance of the next generation of multi-domain joint integrated Command, Control, Communications, Computers, and Intelligence (C4I) and Software Intensive Systems (SIS) systems and architectures. C4T technology development works to emulate joint network operations in a system of systems (SoS) test environment and to analyze and evaluate the increasing mass of structured and unstructured data generated by such tests. To this end, C4T advances technologies for test tools and distributed network infrastructures to support the creation, management, measurement, and analysis of the live-virtual-constructive (LVC) environment, execution of multi-domain joint mission thread tests, and analysis of test results. C4T TTA focuses on developing test technologies for next generation resilient, survivable, federated networks and information ecosystems (information superiority) from the tactical level up to strategic planning. Advances in the warfighter’s C4I/SIS in military operational capabilities are outpacing the current capabilities of the Test & Evaluation (T&E) environment. C4I and SIS are increasing in complexity.

Within the T&E/S&T Program, C4T T&E needs are grouped into three domains:

Command and Control (C2) Analysis in Multi-Domain Environments, Big Data Analytics, and Live and Simulated Environments.

Major test technology needs include the following:

· Ability to reduce T&E infrastructure biases; improve T&E multi-domain operations, cross-domain and multi-level security abilities; advance T&E for C4I/C2 platforms employing big data and tactical cloud environments; and advance T&E for C4I/C2 platforms that operate in agile, contested, and dense communication environments.

· Modeling and simulation (M&S) capabilities representing multi-domain operations (MDO) and the systems, communications, and environments to determine required fidelity for live and simulated MDO based tests; improve verification and validation (V&V) and aggregation techniques; improve run-time performance for real-time applications.

· Automated capabilities to advance testing, especially the collection, analysis, reporting, and visualization of data through development of a common foundation/framework of shared and reusable data, tools, frameworks and standards, and cloud and edge services in support of multi-domain operations. In parallel, take steps to ready existing processes for AI/ML systems through digitization and smart automation. Taken together, these enterprise-wide changes promote the spread of adaptable problem-solving T&E tools using big data analytics to achieve the goal of decreased time to plan, execute, report on joint mission multi-domain events.

· Automated technologies to assess the next generation of C4I/C2 warfighter information systems implementing AI/ML supporting a cross-domain kill web.

· Methods and technologies are needed to improve the use of distributed testing to assess new C4I/C2 warfare systems against emerging/operational threats.

COMMAND, CONTROL, COMMUNICATIONS, COMPUTERS, AND INTELLIGENCE (C4I) AND SOFTWARE INTENSIVE SYSTEMS TEST (C4T)

A-12

Cyberspace Test Technology (CTT) led by the U.S. Air Force (USAF) 96th Cyberspace Test Group, Eglin Air Force Base (AFB), FL. CTT develops advanced technologies and methodologies to test and evaluate DoD’s offensive and defensive military operations across cyberspace. The CTT’s vision is to develop advanced technologies and methodologies to enable DoD components to perform testing as early as possible in the acquisition and accreditation process and to continue that testing throughout the lifecycle of those systems.

Within the T&E/S&T Program, CTT T&E needs are grouped into three domains:

Cyber-Physical Systems that include kinetic systems, cyber-physical networks, and embedded systems. An example is a computer system with a dedicated function within a larger mechanical or electrical system, and often with real-time computing constraints.

Tactical Edge Networks that include information systems and connectivity supporting tactical edge communications and distributed operations, including line-of-sight and beyond-line-of-sight data links, and other networked systems in the battlespace.

Enterprise Information Systems that include the broad scope of unified communications and integration of telecommunications, computers, necessary enterprise software, middleware, storage, and audio-visual systems, which enable users to access, store, transmit, and manipulate information.

Major test technology and methodology needs include the following:

· Measure & Share: Tools and methodologies to measure and document the efficacy of cyber test events with mechanisms that are consistent across the DoD, and that enable sharing test results. This includes methods and tools that are based on sound scientific principles to quantify and qualify the results of cyber test events of all sizes from very small tests on a single standalone piece of hardware to very large test events such as a fleet exercise. Also included are tools and methods to enable sharing of test results of all DoD test events while still protecting sensitive information using techniques such as anonymization.

· DevSecOps: Tools and methodologies to verify secure coding techniques are used that can be applied through the entire development, deployment, and software lifecycle processes.

· Cyber Range Testing: Tools and methodologies that can be applied throughout the entire development, deployment, and software lifecycle to verify secure coding techniques are used and that cyber systems are reliable. CTT is focusing on fuzzing tools and methods and tools that enable the use of formal methods.

· In-situ Testing: This includes tools for in-situ cyber testing of single components, tools and methodologies for enabling in-situ cooperative vulnerability assessments, especially of ships, aircraft and weapons systems, and tools and methodologies to enable DoD Red Teams to better emulate nation state adversaries.

CYBERSPACE TEST TECHNOLOGY (CTT)

Directed Energy Test (DET) led by the U.S. Army Program Executive Office for Simulation, Training, and Instrumentation (PEO STRI), Orlando, FL. DET develops on-board and off-board technologies to enable quantitative assessments of high energy laser (HEL) and high power microwave (HPM) system performance and energy–target interactions to support thorough testing of U.S. directed energy weapon (DEW) systems and friendly system susceptibility to enemy DEWs. This TTA also considers other non-DE weapons that produce ionizing radiation to disable or damage electronics.

Within the T&E/S&T Program, DET T&E needs are grouped into three primary domains:

High Energy Laser, High Power Microwave, and Special Interest Area (Nuclear Effects).

Major test technology needs include the following:

· Safety and test planning tools for DE tests, both HEL and HPM tests in a test cell or open-air engagements

· Ability to predict DE system performance throughout the kill chain using M&S

· Ability to measure an HEL beam incident on a distant target

· Ability to characterize target response to an HEL beam

· Atmospheric compensation technology to improve imaging of HEL beam profiles and target interactions

· Characterizing the atmosphere prior to and during an HEL open-air test

· Survivability and non-intrusiveness of HEL and HPM instrumentation

· Low-noise, broadband non-intrusive HPM sensors

· Ability to simulate operational, environmental conditions for DE systems

· Ability to emulate threat DE and non-DE systems

· Ability to assess the vulnerability of blue military systems to red HELs

· Ability to generate ionizing effects – without radioactive material – to support system vulnerability testing

DIRECTED ENERGY TEST (DET)

Electronic Warfare Test (EWT), led by U.S. Army PEO STRI, Orlando, FL., develops advanced test and evaluation (T&E) technologies and methodologies for installed system test facility and hardware-in-the-loop (ISTF/HITL) laboratories and open-air ranges (OARs). T&E technologies are broadly partitioned into radio frequency (RF) and electro-optical (EO)/Infrared (IR) systems that include electronic warfare (EW) systems, EO/IR and RF countermeasures; counter – improvised explosive device (C-IED) systems; hostile fire indicator systems; intelligence, surveillance, and reconnaissance (ISR) sensor systems; missile warning sensors; infrared countermeasures (IRCM); and weapon sensor/seeker systems.

Within the T&E/S&T Program, EWT T&E needs are grouped into two domains:

Electro-Optical and Radio Frequency Major test technology needs include the following:

· Improve T&E of EW systems

· Maturation of RF and EO/IR source technology for T&E

· Rapid modification of representative live and simulated RF electronic warfare (EW) threats to align with new threat information and changing engagement tactics

· Enable reconfigurable EO/IR and RF threat simulators

· Missile surrogates and simulators for ground testing of IRCM systems

· Missile surrogates and simulators for OAR testing of IRCM systems

· Simulation of RF and EO/IR targets and environments

· Realistic T&E environments for missile warning systems, RF countermeasure systems, ISR sensor systems, and weapon sensor/seeker systems

· T&E of cognitive enabled systems, such as cognitive radar or jammer systems

· Mitigate EW T&E impacts from 5G spectrum loss

ELECTRONIC WARFARE TEST (EWT)

High Speed Systems Test (HSST) led by the USAF Arnold Engineering Development Complex (AEDC), Arnold AFB, TN. HSST identifies and develops advanced test technologies and methodologies for ground test, flight test, and computational M&S for the T&E of advanced high-speed and hypersonic (HS/H) weapons systems such as hypersonic cruise missiles, boost-glide vehicles, turbine-based and rocket-based combined cycle systems, hypersonic gun-launched projectiles, reentry systems, ballistic missile interceptors, and ISR aircraft along with their associated air-breathing propulsion and thermal protection systems. The goal of this TTA is to develop, validate, and support transition of T&E technologies that will permit effective and timely T&E of advanced HS/H weapon systems with the same accuracy and robustness as legacy subsonic and supersonic systems.

Within the T&E/S&T Program, HSST T&E needs are grouped into five domains: Advanced Propulsion, Aerodynamics and Aerothermal, Flight Test and Evaluation, Instrumentation and Diagnostics for High-Speed Flow Fields, and Computational Tools for High-Speed Flow Fields.

Major test technology needs include the following:

· Development and demonstration of novel airborne sensor suites, technologies, and methodologies for supporting HS/H flight test data collection. This includes data collection capabilities such as telemetry, multispectral imaging, atmospheric sensing, and terminal scoring systems, new operational concepts for improving support to HS/H flight tests, and new airborne platforms to support data collection requirements.

· Development and demonstration of new, flight-representative ground test technologies that enable high-fidelity HS/H system characterization for flight test programmatic and technical risk reduction. This includes technology advancements in ground test facilities that enable clean-air, true temperature test processes with mission duration runtimes, variable Mach number test capabilities, aero-optic and seeker-sensor test methodologies, system aerothermal development, and weather encounter characterization.

· Quantification, characterization, and improvement of ground and flight test methodologies and best practices associated with test configurations (e.g., flight, free-jet, direct-connect, blow-down, and impulse testing), test facility infrastructure and operational processes (e.g., clean, vitiated combustion or arc jet heated air, and quiet wind tunnel flow quality versus traditional test facility; and test duration), and flight test support infrastructure for characterization of hypersonic system aerothermal structures, engine aero propulsion, aero-optics, thermal management, and fuel system performance and operability

· Development and demonstration of ground and flight test technologies and methodologies for the characterization of turbine and rocket-based combined cycle high speed systems, to include test facility infrastructure, instrumentation and diagnostics, test processes, and analysis techniques.

· Advancement of instrumentation, diagnostics, and M&S tools for HS/H system development

HIGH SPEED SYSTEMS TEST (HSST)

Space Systems Test (SST) is currently under instantiation and is led by Test Resource Management Center, Alexandria, Virginia. Current T&E capabilities for space systems focus on system functionality within the space environment, but lack capabilities for full system testing of suitability and survivability in a contested domain. Current capabilities may be adequate to demonstrate functionality in the operational environment, but are inadequate for acquisition programs’ testing in a contested environment. Enhanced T&E capabilities are needed to cover areas such as, but not limited to, the following categories: threat emulation, indications and warnings, problem resolution and attack attribution, TTP development, kill chain response, system of system performance, cyber resiliency, and resiliency of C2 systems. The T&E for acquisition programs must allow for the highly classified nature of space systems and the availability of test articles from the program office. Needs are grouped into three segments common among all space systems:

Space Segment: This is the part of the system located on-orbit. This segment includes the bus and payload that make up a satellite or constellation of two or more satellites designed to work together to perform a common mission (i.e. Global Positioning System).

Ground Segment: This includes the ground-based networks that enable operations of the satellite system, including command and control operations centers and networks, as well as, the equipment required for users to receive/process data provided by the satellite systems.

Link Segment: This is defined by the signals between the different segments, such as between satellites, satellite and user, and satellite and ground segments. This segment includes the hardware required to generate the links.

Within each of these segments, SST uses three technology domains with more focused needs and respective T&E benefits:

Digital Engineering System Modeling, which includes representing end-to-end threat engagements of the Space, Ground, or Link segments;

Non-kinetic Threat Emulation, which addresses Cyber and EW attack on Space, Ground, and Link segments of space systems; and Kinetic Threat Emulation, which supports Direct Assent (DA) and Co-Orbital Kinetic Kill Vehicle attack.

SST supports five strategic DoD objectives for instantiating T&E/S&T capabilities addressing a contested space domain, as follows:

· Build up integrated test environments to enhance live and virtual tests to assess performance and survivability of space systems

· Advance test capability and capacity to verify functionality, suitability, and effectiveness of blue force systems

· Advance threat emulation to understand impacts to, mitigation strategies, and survivability of blue force space systems

· Enable innovative solutions for key T&E gaps from external partners, such as industry, labs, academia, etc.

· Build up foundational T&E infrastructure to enable workforce growth and training, data management, and cyber protection for enduring mission capabilities

SPACE SYSTEMS TEST (SST)

Spectrum Efficient Technology (SET) is led by the U.S. Air Force Test Center, Edwards AFB, CA. SET develops test technologies and methodologies to enable more efficient use of legacy telemetry spectrum bands, support implementation of a robust networked telemetry capability, and support expansion into non-traditional areas of the RF and optical spectrum at DoD test ranges. SET focuses on addressing the growing data requirements for testing warfighting systems and the limited availability of RF spectrum for wirelessly collecting test data. The goal of this TTA is to develop the technologies required to ensure that the MRTFB and other test ranges have the telemetry capability necessary to conduct quality testing on each new generation of military systems. The TTA advances hardware and software technologies required to improve spectral efficiency and enable OAR wireless technology.

Within the T&E/S&T Program, SET T&E needs are grouped into three domains:

Wireless Technologies, Telemetry Networking Technologies, and Spectrum Management.

Major test technology needs include the following:

· Ability to advance range communications, especially wireless communications

· Ability to enhance networked telemetry capabilities

· Improved usage of the RF spectrum for effective management of the complex telemetry environment at an OAR

· Next-generation spectrum planning tools that provide dynamic frequency re-allocation