DARPA-PA-21-04-05.pdf

DARPA-PA-21-04-05

Artificial Intelligence Exploration (AIE) Opportunity Pre-Solicitation Number DARPA PA-21-04-05

Geometries of Learning (GoL)

I. Opportunity Description The Defense Advanced Research Projects Agency (DARPA) is issuing an Artificial Intelligence Exploration (AIE) Opportunity inviting submissions of innovative basic or applied research concepts in the technical domain of high-dimensional geometric analysis of deep neural networks (DNNs) applied to image data.

The Geometries of Learning (GoL) AIE Opportunity aims to advance the theory of Deep Learning (DL) by better understanding the geometry of natural images in image space, the geometry of functions that map from image space to feature and label space, and the geometry of how these mapping functions evolve during training.

This AIE Opportunity is issued under the Program Announcement for AIE, DARPA-PA-21-04.

All awards will be made in the form of an Other Transaction (OT) for prototype project. The total award value for the combined Phase 1 base (Feasibility Study) and Phase 2 option (Proof of Concept) is limited to $1,000,000. This total award value includes Government funding and performer cost-share, if required or if proposed.

To view the original DARPA Program Announcement for the AIE Opportunity visit beta.SAM.gov under solicitation number DARPA-PA-21-04:

https://sam.gov/opp/b04934de80af4f5b92ab51852266c8c7/view

A. Introduction The Department of Defense is committed to the rapid adoption of machine learning technology1.

Unfortunately, the practice of DL has outpaced the theory, creating barriers to adoption. The underlying hypothesis is that the set of images of an object form a manifold in image space, which in turn can be used to restrict the mapping from image space to label space and improve the training process for DNNs.

A better understanding of the geometries of learning is expected to yield practical insights into many difficult problems. Examples include:

Adversarial AI: Adversarial examples are images or scenes that have been modified by an adversary to fool deep networks (see the DARPA Guaranteeing AI Robustness against Deception (GARD) program at https://www.darpa.mil/program/guaranteeing-ai-robustness-against-deception). One hypothesis about why deep networks are vulnerable to adversarial attacks is that they are forced to map all image space to label space, even though most the image space contains images of random noise. If we can understand the

1 National Security Commission on Artificial Intelligence. (2021, January). Final Report.

https://www.nscai.gov/wp-content/uploads/2021/03/Full-Report-Digital-1.pdf https://sam.gov/opp/b04934de80af4f5b92ab51852266c8c7/view manifolds natural images form in image space, we can potentially restrict the domain of the function, making neural nets less susceptible to adversarial attacks.

Explainable AI: It is important for human operators to understand the basis for the output of a DNN, beyond the label the network assigns to an image (see DARPA Explainable Artificial Intelligence (XAI) program at https://www.darpa.mil/program/explainable-artificial-intelligence). If we understand the manifold created by the images of an object, then the position of an image on the manifold conveys additional information, such as the pose of the object or how the scene is illuminated. Instead of producing just a label (e.g., “cat”), deep networks might provide descriptions (e.g., “cat, laying down, viewed from the side”).

Trainable AI: A barrier to machine learning in many applications is a lack of large enough training sets (see DARPA Learning with Less Labeling (LwLL) program at https://www.darpa.mil/program/learning-with-less-labeling). Once again, if we understand the geometry of image manifolds, we can determine whether we have enough samples to model an object, and if not, what samples we need to add.

Trustworthy AI: Operators need to know when to trust machine learning systems and when not to. One reason to distrust a deep network is the stability of the learning process.

If the insertion or removal of a few natural samples from the training set significantly impacts the decisions of the net, then those decisions may not be reliable.

GoL will exploit the classic observation that naturally occurring images lie on manifolds occupying only a tiny portion of image space2, creating the opportunity for modern analysis approaches to be restricted to these manifolds. If successful, this will improve our theoretical understanding of issues related to adversarial AI, explainable AI, trainable AI, and trustworthy

AI.

B. Objective/Scope The objective of the GoL AIE is to develop high-dimensional geometric techniques and tools for understanding the behavior of DNNs applied to images. Current DNNs define mappings from image space to label space, including the vast swathes of image space that contain only noise patterns. By restricting the domain of functions learned by DNNs to object manifold in image space, we hope to make neural nets more robust, more interpretable, and more easily analyzed.

Of specific interest to GoL are (1) modeling the manifold geometry of images in input image space; (2) modeling the non-linear input-to-output mapping learned by a DNN restricted to these manifolds; and (3) analyzing the uniqueness and stability of the mappings learned by DNNs over image space manifolds.

2 Murase, Hiroshi, and Shree K. Nayar. "Visual Learning and Recognition of 3-D Objects from Appearance."

International Journal of Computer Vision. 14.1 (1995): 5-24.

Imaging combines properties of objects (e.g., their shape, color, and pose) with scene properties (e.g., illumination) and sensor parameters (e.g., aperture) in a complex, non-linear process to create natural images. Some object properties are fixed, for example, few objects dynamically change color. Other properties change smoothly. A rigid object’s pose can vary in six dimensions (three for orientation, three for position), while the shape of articulated, flexible, and fluid objects can change along additional dimensions. Scene and sensor properties are similar in that they change the image in non-linear but differentiable ways along known dimensions, such as the intensity, color, and location of the illumination source. As a result, the set of all possible images of an object forms a continuous manifold in image space. This manifold, although high-dimensional, is still lower-dimensional than the ambient image space it is defined in. In fact, most of image space contains meaningless images of noise. Only small parts of image space contain images of natural objects, as defined by their manifolds.

GoL is anticipated to be a fundamental research effort. Phase 1 will focus on theory development and a feasibility study. Phase 2 will focus on theory refinement and delivery of a proof of concept. Proposers are encouraged to build upon and expand existing theory and contribute back to the open-source community if software tools are developed. Proposers should address how their proposals are well-grounded with existing theory and tools, but also provide a plan for applying theory to a major AI challenge.

C. Structure Proposals submitted to DARPA-PA-21-04-05 in response to the technical area of this AIE Opportunity must be UNCLASSIFIED and must address two independent and sequential project phases (a Phase 1 Feasibility Study (base) and a Phase 2 Proof of Concept (option)). The periods of performance for these phases are nine (9) months for the Phase 1 base effort and nine (9) months for the Phase 2 option effort. Combined Phase 1 base and Phase 2 option efforts for this AIE Opportunity should not exceed 18 months. The Phase 1 (base) award value is limited to $500,000. The Phase 2 (option) award value is limited to $500,000. Both Phase 1 and Phase 2 value limits include performer cost-share, if required or if proposed. The total award value for the combined Phase 1 and Phase 2 is limited to $1,000,000. This total award value includes Government funding and performer cost-share, if required or if proposed.

D. Schedule/Milestones Proposers must address the following fixed payable milestones in their proposals. If selected for award negotiation, the fixed payable milestones provided will be directly incorporated into Attachment 3 of the OT agreement (“Schedule of Milestones and Payments”). Proposers are encouraged to use the Task Description Document template provided with the Program Announcement DARPA-PA-21-04, which will be Attachment 1 of the OT agreement.

Phase 1 fixed milestones for this program must include, at a minimum, the following, delivered as Milestone reports and, in the case of demonstrations, accompanied by relevant software artifacts:

(Month 2): Milestone 1 report outlining a research and development roadmap for theoretical tools by the end of GoL Phase 1, combined with a rough plan to put theory into practice against at least one major AI challenge (e.g., adversarial, explainable, trainable, or trustworthy AI) related to better understanding the geometries of learning by the end of Phase 2. This report should address the current literature and theoretical underpinnings GoL proposers intend to build upon. The report should clearly outline what GoL will be adding to it. In the more likely scenario where a current theory does not exist, the proposer should discuss in sufficient detail specific foundations for the theory to be further explored and developed during Phase 1 of the program.

(Month 6): Milestone 2 report to include an initial development of the theory to be finalized by the end of Phase 1. To the extent that the theory or approach has changed from the previous milestone (e.g., with respect to the AI problem or challenge defined in Milestone 1), this should be documented.

(Month 9): Milestone 3 Phase 1 report and final demonstration of the theory and how that will be further refined and applied in Phase 2. Report on Phase 2 plans to include a timeline for the development and application of theoretical tools. If software tools are appropriate to help apply the theory to a task (e.g., software to fit a manifold to data or to estimate the dimensionality of a manifold), plans for the development of these tools should be described.

Phase 2 fixed milestones for this program must include, at a minimum, the following, delivered as Milestone reports and, in the case of theoretical tools, accompanied by relevant software artifacts:

(Month 11): Milestone 4 report outlining how the theory developed in Phase 1 (and possibly being extended in Phase 2) applies to one of the challenges or adversarial, explainable, trainable, or trustworthy AI. This must include a plan or roadmap for making major leaps in solving an AI problem by the end of GoL Phase 2. A plan for software tools to help apply the theory to real problems is recommended but not required.

(Month 15): Milestone 5 report on the current state of the developing theory and how it applies to one or more of the challenge problems. To the extent that the approach has changed from the previous milestone(s), this should be documented. If software tools are being developed as part of this theory, the report must include performer plans to open source their technology.

(Month 18): Milestone 6 final report describing the theory developed under the GoL program and explaining how it applies to one of the four identified challenges of AI. If software was developed as part of GoL, the report must point to where the software has been open-sourced.

For planning and budgetary purposes, proposers should assume a program start date of April 4, 2022. Schedules will be synchronized across performers, as required, and monitored/revised as necessary throughout the program.

All proposals must include the following meetings and travel in the proposed schedule and costs:

To foster collaboration between teams and disseminate program developments, a one-day Principal Investigator (PI) meeting will be held approximately every six months, with locations split between the East and West Coasts of the United States. For budgeting purposes, plan for three two-day meetings over the course of 18 months: two meetings in the Washington, D.C.

area and one meeting in the San Francisco, CA area. Regular teleconference meetings will be scheduled with the Government team for progress reporting as well as problem identification and mitigation. Proposers should also anticipate at least one site visit, likely virtual, per phase by the DARPA Program Manager, during which they will have the opportunity to demonstrate progress towards agreed-upon milestones.

E. Deliverables In addition to the milestone reports, performers will be expected to provide the following deliverables:

Negotiated deliverables specific to the objectives of the individual efforts. These may include registered reports, experimental protocols, publications, intermediate and final versions of software libraries, code, and APIs, including documentation and user manuals, and/or a comprehensive assemblage of design documents, models, modeling data and results, and model validation data.

II. Award Information Selected proposals that are successfully negotiated will result in award of an OT for prototype project. See Section 3 of DARPA-PA-21-04 for information on awards that may result from proposals submitted in response to this notice.

Proposers must review the model OT for Prototype agreement provided as an attachment to DARPA-PA-21-04 prior to submitting a proposal. DARPA has provided the model OT in order to expedite the negotiation and award process and ensure DARPA achieves the goal of AIE, which is to enable DARPA to initiate a new investment in less than 90 calendar days from idea inception. The model OT is representative of the terms and conditions that DARPA intends to award for all AIE Awards. The task description document, schedule of milestones and payments, and data rights assertions requested under Volumes 1, 2, and 3 will be included as attachments to the OT agreement upon negotiation and award.

Proposers may suggest edits to the model OT for consideration by DARPA and provide a copy of the model OT with track changes as part of their proposal package. DARPA may not accept suggested edits. The Government reserves the right to remove a proposal from award consideration should the parties fail to reach an agreement on OT award terms and conditions. If edits to the model OT are not provided as part of the proposal package, DARPA assumes that the proposer has reviewed and accepted the award terms and conditions to which they may have to adhere and the sample OT agreement provided as an attachment, indicating agreement (in principle) with the listed terms and conditions applicable to the specific award instrument.

To ensure that DARPA achieves the AIE goal of award within 90 calendar days from the posting date (January 5, 2022) of this announcement, DARPA reserves the right to cease negotiations when an award is not executed by both parties (DARPA and the selected organization) on or before April 4, 2022.

III. Eligibility

See Section 4 of the AIE Program Announcement DARPA-PA-21-04 for information on who may be eligible to respond to this notice.

IV. AIE Opportunity Responses Responses to this AIE opportunity must be submitted as full proposals to DARPA-PA-21-04-05 as described therein. All proposals must be unclassified.

A. Proposal Content and Format All proposals submitted in response to this notice must comply with the content and format instructions in Section 5 of DARPA-PA-21-04. All proposals must use the templates provided as Attachments to the PA and the “Schedule of Milestones and Payments” Excel Attachment provided with this AIE Opportunity and follow the instructions therein.

Information not explicitly requested in DARPA-PA-21-04, its Attachments, or this notice may not be evaluated.

B. Proposal Submission Instructions Responses to DARPA-PA-21-04-05 shall be submitted electronically to DARPA’s Broad Agency Announcement (BAA) Portal (https://baa.darpa.mil).

DARPA will acknowledge receipt of complete submissions via email and assign identifying numbers that should be used in all further correspondence regarding those submissions. If no confirmation is received within two (2) business days, please contact GoL@darpa.mil to verify receipt.

When planning a response to this AIE Opportunity, proposers should take into account the submission time zone and that some parts of the submission process may take from one business day to one month to complete (e.g., registering for a Data Universal Numbering System (DUNS) number or Tax Identification Number (TIN)).

Electronic Upload

First-time users of the DARPA BAA Portal must complete a two-step account creation process.

The first step consists of registering for an extranet account by going to the URL listed above and selecting the “Account Request” link. Upon completing the online form, proposers will receive two separate emails; one will contain a user name, and the second will provide a temporary password. Once both emails have been received, the second step requires proposers to go back to the submission website and log in using that user name and password. After accessing the extranet, proposers may then create a user account for the DARPA Submission website by selecting the “Register your Organization” link at the top of the page. Once the user account is created, proposers will see a list of solicitations open for submissions, view submission instructions, and upload/finalize their proposal.

Proposers who already have an account on the DARPA BAA Portal may simply log in at https://baa.darpa.mil/ mailto:GoL@darpa.mil https://baa.darpa.mil, select this solicitation from the list of open DARPA solicitations and proceed with their proposal submission. Note: proposers who have created a DARPA Submission website account to submit to another DARPA Technical Office’s solicitations do not need to create a new account to submit to this solicitation. However, accounts are typically disabled and eventually deleted following 75-90 days of inactivity – if you are unsure when the account was last used, it is recommended that you create a new account.

All full proposals submitted electronically through the DARPA Submission website must meet the following requirements: (1) uploaded as a zip file (.zip or .zipx extension); (2) only contain the document(s) requested herein; (3) only contain unclassified information; and (4) must not exceed 100 MB in size. Only one zip file will be accepted per full proposal. DARPA will reject full proposals not uploaded as zip files. Technical support for the DARPA Submission website is available during regular business hours, Monday – Friday, 9:00 a.m. – 5:00 p.m. Requests for technical support must be emailed to BAAT_Support@darpa.mil with a copy to GoL@darpa.mil. Questions regarding submission contents, format, deadlines, etc., should be emailed to GoL@darpa.mil. Questions/requests for support sent to any other email address may result in delayed/no response.

Since proposers may encounter heavy traffic on the web server, DARPA discourages waiting until the day proposals are due to request an account and/or upload the submission. Note:

proposers submitting a proposal via the DARPA Submission site MUST: (1) click the “Finalize” button in order for the submission to upload AND; (2) do so with sufficient time for the upload to complete prior to the deadline. Failure to do so will result in a late submission.

C. Proposal Due Date and Time Proposals in response to this notice are due no later than 4:00 PM on February 3, 2022. As described in Section 5 of DARPA-PA-21-04, full proposal packages must be submitted per the instructions outlined in this AIE Opportunity and received by DARPA no later than the above time and date. Proposals received after this time and date may not be reviewed.

Proposers are warned that the proposal deadline outlined herein is in Eastern Time and will be strictly enforced. When planning a response to this notice, proposers should take into account that some parts of the submission process may take from one (1) business day to one (1) month to complete.

I. Proposal Evaluation and Selection Proposals will be evaluated and selected in accordance with Section 6 of DARPA-PA-21-04.

Proposers will be notified of the results of this process as described in Section 7.1 of DARPA-

PA-21-04.

II. Administrative and National Policy Requirements Section 7.2 of DARPA-PA-21-04 provides information on Administrative and National Policy Requirements that may be applicable for proposal submission as well as performance under an award.

https://baa.darpa.mil/

III. Point of Contact Information Bruce Draper, Program Manager, DARPA/I2O, GoL@darpa.mil

IV. Frequently Asked Questions (FAQs) All technical, contractual, and administrative questions regarding this notice must be emailed to GoL@darpa.mil. Emails sent directly to the Program Manager or any other address may result in delayed or no response.

All questions must be in English and must include the name, email address, and telephone number of a point of contact. DARPA will attempt to answer questions publicly in a timely manner; however, questions submitted within seven (7) calendar days of the proposal due date listed herein may not be answered.

DARPA will post an FAQ list under the AIE Opportunity on the DARPA/I2O Opportunities page at http://www.darpa.mil/work-with-us/opportunities. The list will be updated on an ongoing basis until one (1) week prior to the proposal due date. In addition to the FAQ specific to this notice, proposers should also review the Program Announcement for AIE General FAQ list on the DARPA/I2O Opportunities page under the Program Announcement for AIE (DARPA-PA- 21-04).

mailto:GoL@darpa.mil http://www.darpa.mil/work-with-us/opportunities