TrojAI

Response Dates:
Concept Papers- May 31 2019 no later than 4:00 PM Eastern Time (Must Submit)
Selection of Concept Papers for full proposal on June 10th 2019
Proposals Due - July 25 2019 no later than 4:00 PM Eastern Time
The application process under this BAA consists of a Concept Paper stage and a Proposal stage. The purpose of this two-step approach is to facilitate pre-screening by the U.S. Government such that detailed proposals are only sought from applicants whose concept papers demonstrate the most promise for award (this also helps to reduce unnecessary bid and proposal effort). The government's decision to invite a Proposal will be based upon the evaluation results of a timely and compliant Concept Paper submission. Only the most highly rated Concept Papers will receive an invitation from the government to submit a Proposal. An Applicant that does NOT submit a timely and compliant Concept Paper, is NOT eligible to submit a Proposal for consideration for funding. An Applicant that does NOT receive an invitation from the Government to submit a Proposal is NOT eligible to submit a Proposal. An Applicant invited to submit a Proposal will receive feedback on their Concept Paper that is expected to improve their Proposal submissions. To facilitate organizations who might want to come together to submit one proposal, all proposers invited for full proposal will receive each others' contact details and concept papers (sans budget section).
Program Overview
The U.S. Army Research Office (ARO) in partnership with the Intelligence Advanced Research Projects Activity (IARPA) seeks research and development of technology and techniques for detection of Trojans in Artificial Intelligence. TrojAI is envisioned to be a 2-year effort with multiple awardees coming together as a group of performers (hereinafter referred to as the "performer team"), which will work together to achieve the common program goals set forth in this BAA.
Using current machine learning methods, an artificial intelligence (AI) is trained on data, learns relationships in that data, and then is deployed to the world to operate on new data. For example, an AI can be trained on images of traffic signs, learn what stop signs and speed limit signs look like, and then be deployed as part of an autonomous car. The problem is that an adversary that can disrupt the training pipeline can insert Trojan behaviors into the AI. For example, an AI learning to distinguish traffic signs can be given potentially just a few additional examples of stop signs with yellow squares on them, each labeled "speed limit sign". If the AI were deployed in a self-driving car, an adversary could cause the car to run through the stop sign just by putting a sticky note on it, since the AI would incorrectly see it as a speed limit sign. The goal of the TrojAI Program is to combat such Trojan attacks by inspecting AIs for Trojans.
The performer team will develop and deliver software to automatically inspect an AI and predict if it has a Trojan. The AIs will be neural networks trained in a classification task. Initially, the AIs will classify small images, but later stages of the program may expand to AIs that classify audio or text or perform other tasks. The AIs will classify input data into a small number of classes at near-human or super-human performance. As a reference, the image classification tasks will be analogous to the German Traffic Sign Recognition, but performers should not assume that the input data or classes correspond to any public data sets. Trojan attacks will modify a portion of the AIs to recognize input triggers and cause misclassification, similar to the "speed limit sign" example above. The performer team will deliver software that detects which AIs have been subject to a Trojan attack. Performer software will take as input the AI's source code, architecture and compiled binary. The performer software may also receive a small number of examples of valid data from the AI's problem domain. The performer team should be prepared to initially develop methods for AIs made of feedforward networks (e.g. Convolutional Neural Nets like ResNet for image classification) and then later develop methods for AIs made of recurrent networks (e.g. Long-Short Term Memory Networks for audio or text classification).

.