Securing Artificial Intelligence for Battlefield Effective Robustness (SABER)

There is a growing desire to integrate rapidly advancing artificial intelligence (AI) technologies into Department of Defense (DoD) systems. AI may give battlefield advantage by helping improve the speed, quality, and accuracy of decision-making while enabling autonomy and assistive automation. Due to the statistical nature of machine learning, significant work has focused on ensuring the robustness of AI-enabled systems at inference time to natural degradations in performance caused by data distribution shifts (for example, from a highly dynamic deployment environment). However, as early as 2014, researchers demonstrated the ability to manipulate AI given adversary control of the input[1]. Additional work has further confirmed the theoretical risks of data poisoning[2], physically constrained adversarial patches for evasion[3], and model-stealing attacks[4]. These attacks are typically tested in simulated or physical environments with relatively pristine control compared to what might be expected on a battlefield. Today, there is still a limited ability to operationally assess deployed military AI-enabled systems for adversarial vulnerabilities and the “theoretical” adversarial AI attacks have not been practically demonstrated in operational settings. As a result, the operational security risks of AI-enabled battlefield systems remain largely unknown.

SABER aims to build an exemplar AI red team equipped with the necessary counter-AI techniques, tools, and technical competency to operationally assess AI-enabled battlefield systems. SABER seeks to establish a sustainable model for an operational AI red teaming process for the DoD. Our AI red team will target operationally assessing AI-enabled autonomous ground and aerial systems that could be deployed within the next 1-3 years. To assist the AI red team, the future solicitation will seek performers who can assist in surveying, evaluating, selecting, developing, and employing state-of-the-art physical (including manufacturing/materials), adversarial AI (including digital), cyber, and electronic warfare techniques and tools, or other relevant vectors for operational assessment of AI-enabled battlefield system development and deployment pipelines[5]. Additionally, the forthcoming solicitation will seek a performer to serve as an integration lead, assisting in integrating the technologies into an interoperable, operational AI red teaming toolkit to enable future AI red teams for the DoD and broader U.S. Government.

Citations:
1 https://arxiv.org/abs/1412.6572
2 https://arxiv.org/abs/1804.00792
3 https://arxiv.org/pdf/2104.06728
4 https://arxiv.org/abs/2411.10023
5 https://sam.gov/opp/041e8bd594c1450dbec14a6ca580fbed/view