Sbir phase I: post-training deep neural networks certification against backdoor data poisoning attacks

This $255,391 National Science Foundation Technology, Innovation, and Partnerships grant funds Anomalee Inc. to develop a commercially viable prototype for certifying deep neural networks against backdoor attacks. Backdoor attacks maliciously poison AI models to misclassify inputs containing hidden trigger patterns. Anomalee's unsupervised method will mathematically evaluate DNNs for such vulnerabilities without examples of poisoned models or access to training data, addressing a critical need as DNNs increasingly automate safety and security functions. The April 2022 through December 2022 project aims to make the detector insensitive to backdoor incorporation mechanisms across domains and data types, including images, text, time-series, and 3D data. APIs will expand capabilities to related attacks. This award advances the NSF's mission to support innovative research addressing national challenges through merit-based peer review.

Abstract

The broader impact of this small business innovation research (SBIR) phase I project will be to secure and certify deep learning models that are becoming ubiquitous in many safety and security-sensitive applications, such as finance, health, military/intelligence, cyber security, critical infrastructure, and personal/consumer use. Strong growth in deep neural network (DNN) deployments is forecast in the near term in several of these domains, some of which are subject to regulatory requirements that artificial intelligence (AI) models be certified to perform as advertised. This project proposes a new method to confidently certify against backdoor attacks. This small business innovation research phase I project will provide the first commercial prototype of a mathematically principled certification service for dnns against evasive backdoor attacks (BAS). The proposed method is wholly unsupervised, requiring no known examples of poisoned dnns nor any samples from the (possibly poisoned) training set. This project will advance a broadly applicable and computationally efficient approach through parallel computation leveraging cost-effective cloud-computing services, to address challenges such as very large input feature space dimensions and number of classes, as well as very large dnns. Another challenge is to make the detector insensitive to the mechanism (e.g., additive, multiplicative) by which the backdoor pattern (BP) is incorporated into a sample across different application domains. In addition to static dnns and image domains, the prototype will be able to: process recurrent dnns; handle time series, point cloud, and document data domains; and defend AIS used for time-series prediction and regression. Apis will be developed to expand the prototype to defend against related attacks, e.g., backdoor patterns that are perceptible but scene plausible or test-time evasion attacks. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.