HR001117S0016-Amendment-01.pdf

HR001117S0016

Microsystems Technology Office Broad Agency Announcement

Lifelong Learning Machines (L2M)

HR001117S0016

April 12, 2017

Amendment 1 Amended on June 8, 2017

Table of Contents

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

A. Background and Goals B. Program Philosophy and Considerations C. Technical Areas D. System Development Environments and Evaluation (TA1 only) E. Program Schedule and Milestones F. Deliverables

II. Award Information A. General Award Information B. Fundamental Research

III. Eligibility Information A. Eligible Applicants

1. Federally Funded Research and Development Centers (FFRDCs) and Government Entities

2. Non-U.S. Organizations and/or Individuals B. Organizational Conflicts of Interest C. Cost Sharing/Matching D. Collaborative Efforts

IV. Application and Submission Information A. Address to Request Application Package B. Content and Form of Application Submission

1. Abstract Format

2. Full Proposal Format

3. Proprietary Information

4. Security Information

a. Unclassified Submissions

b. Classified Submissions

5. Human Research Subjects/Animal Use

6. Approved Cost Accounting System Documentation

7. Section 508 of the Rehabilitation Act (29 U.S.C. § 749d)/FAR 39.2

8. Grant Abstract

9. Small Business Subcontracting Plan

10. Intellectual Property

a. For Procurement Contracts

b. For All Non-Procurement Contracts

11. Patents

12. System for Award Management (SAM) and Universal Identifier Requirements

13. Funding Restrictions

C. Submission Information

1. Submission Dates and Times

a. Abstract Due Date

b. Full Proposal Date

c. Frequently Asked Questions (FAQ)

2. Abstract Submission Information

3. Proposal Submission Information

a. For Proposers Requesting Grants or Cooperative Agreements:

b. For Proposers Requesting Contracts or Other Transaction Agreements

c. Classified Submission Information

4. Other Submission Requirements

V. Application Review Information A. Evaluation Criteria

1. Overall Scientific and Technical Merit

2. Potential Contribution and Relevance to the DARPA Mission

3. Cost Realism

4. Plans and Capability to Accomplish Technology Transition

B. Review and Selection Process

1. Review Process

2. Handling of Source Selection Information

3. Federal Awardee Performance and Integrity Information (FAPIIS)

VI. Award Administration Information A. Selection Notices

1. Abstracts

2. Proposals

B. Administrative and National Policy Requirements

1. Meeting and Travel Requirements

2. FAR and DFARS Clauses

3. Controlled Unclassified Information (CUI) on Non-DoD Information Systems

4. Representations and Certifications

5. Terms and Conditions

C. Reporting D. Electronic Systems

1. Wide Area Work Flow (WAWF)

2. i-Edison

VII. Agency Contacts VIII. Other Information

A. Protesting

ATTACHMENT 1: Cost Volume Proposer Checklist ATTACHMENT 2: Proposal Summary Slide Template

PART I: OVERVIEW INFORMATION

Federal Agency Name: Defense Advanced Research Projects Agency (DARPA), Microsystems Technology Office (MTO)

Funding Opportunity Title: Lifelong Learning Machines (L2M) Announcement Type: Initial Announcement Funding Opportunity Number: HR001117S0016 Catalog of Federal Domestic Assistance Numbers (CFDA): 12.910 Research and

Technology Development Dates: (All times listed herein are Eastern Daylight Time) o Posting Date: April 12, 2017 o Proposers Day: March 30, 2017 o Abstract Due Date: May 3, 2017 at 1:00PM o FAQ Submission Deadline: June 14, 2017 at 1:00PM (see Section IV.C.1.c, “Frequently Asked Questions (FAQ)”) o Proposal Due Date: June 30, 2017 at 1:00PM o Estimated period of performance start: November 1, 2017

Concise description of the funding opportunity: DARPA is soliciting highly innovative research proposals for the development of fundamentally new machine learning approaches that enable systems to learn continually as they operate and apply previous knowledge to novel situations. Current AI systems only compute with what they have been programmed or trained for in advance; they have no ability to learn from data input during execution time, and cannot adapt on-line to changes they encounter in real environments. The goal of Lifelong Learning Machines (L2M) is to develop substantially more capable systems that are continually improving and updating from experience.

Anticipated individual awards: Multiple awards are anticipated.

Anticipated funding type: 6.1 Types of instruments that may be awarded: Procurement contract, grant, cooperative agreement or other transaction Agency contact:

o Technical POC: Dr. Hava Siegelmann, Program Manager BAA Email: HR001117S0016@darpa.mil BAA Mailing Address:

DARPA/MTO

ATTN: HR001117S0016

675 North Randolph Street Arlington, VA 22203-2114 mailto:name@darpa.mil

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

DARPA is soliciting highly innovative research proposals for the development of fundamentally new machine learning approaches that enable systems to learn continually as they operate and apply previous knowledge to novel situations. Current artificial intelligence (AI) systems only compute with what they have been programmed or trained for in advance; they have no ability to learn from data input during execution time, and cannot adapt on-line to changes they encounter in real environments. The goal of the Lifelong Learning Machines (L2M) program is to develop substantially more capable systems that are continually improving and updating from experience.

Proposed research should investigate innovative approaches that support key lifelong learning machines technologies and enable revolutionary advances in the science of adaptive and intelligent systems. Specifically excluded is research that results in incremental improvements to the existing state of practice.

This BAA is being issued, and any resultant selection will be made, using the procedures under Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016 and 2 C.F.R. § 200.203. Any negotiations and/or awards will use procedures under FAR 15.4, Contract Pricing. Proposals received as a result of this BAA shall be evaluated in accordance with evaluation criteria specified herein through a scientific review process.

DARPA BAAs are posted on the Federal Business Opportunities (FedBizOpps) website, https://www.fbo.gov/, and, as applicable, the Grants.gov website at http://www.grants.gov/.

The following information is for those wishing to respond to this BAA.

A. Background and Goals

Machine learning (ML) methods have demonstrated outstanding recent progress and, as a result, artificial intelligence (AI) systems can now be found in myriad applications, including autonomous vehicles, industrial applications, search engines, computer gaming, health record automation, and big data analysis. At the same time, current ML systems are not intelligent in the biological sense. They have no ability to adapt their methods beyond what they were prepared for in advance and are completely incapable of recognizing or reacting to any element, situation or circumstance they have not been specifically programmed or trained for. This issue presents severe limitations in system capability, creates potential safety issues, and is clearly limiting in Department of Defense (DoD) applications, e.g., supply chain, logistics, and visual recognition, where complete details are often unknown in advance and the ability to react quickly and adapt to dynamic circumstances is of primary importance.

Figure 1: The Benefit of L2M Systems

Current ML systems experiencing errors when they encounter circumstances outside their programming and/or training must be taken off-line and re-programmed/retrained. Taking a system offline and re-training it is expensive and time-consuming, not to mention that encountering a programming/training oversight during execution time can be disruptive to a mission. Current ML systems are also plagued with another significant problem known as catastrophic forgetting. These systems ‘forget’ previously incorporated data when trained with new data and unless programmed or trained for every eventuality, these systems operating in real-world environments are bound to fail at some point. This means ML is restricted to specific situations with narrowly predefined rule sets.

The goal of the L2M program is to develop fundamentally new machine learning mechanisms that enable systems to learn continuously during execution and apply previously learned information to novel situations the way biological systems do and in which the environment is, in effect, the training set. Such a system is safer, more functional, and increasingly relevant to DoD applications, including adapting quickly to unforeseen circumstances, changing the mission, and improving performance through a system’s fielded lifetime experience (see Figure 1).

While it is very easy to code agent behavior to perform a particular task, doing so precludes the agent learning the task, which in turn precludes the possibility of adapting the behavior to another task or situation. This is the heart of the problem to be solved in the creation of a lifelong learning machine. It may be very tempting for proposers to utilize existing techniques to achieve particular behaviors. We are all aware that seemingly complex behavior can be hard coded with often impressive results. However, this is entirely antithetical to the purpose and goals of the L2M program, where the behavior itself (particularly to begin with) is secondary to developing a system that figures out how to accomplish a task and subsequently can figure out another task more easily based on previous learning. The latter is much more difficult to accomplish, and this is exactly what the L2M program is all about.

B. Program Philosophy and Considerations

The learning proposed by the L2M program necessitates innovations: instead of providing the system with a wide knowledge set of the application, as is currently done, an L2M system requires only limited domain knowledge and a way to start its behavior and learning. Following are some brief notes on theory, challenges, and the biology that underlies the L2M system.

State-of-the-art (SOA) computer science follows the 1936 Turing computational paradigm. Most saliently, systems based on Turing machines are limited to exactly what’s written in the program fed to them prior to execution time. Current machine learning enables a limited form of 'learning' during a well-designed preparatory training phase. These systems are still bound by their inability to execute anything beyond their programming and training. A lifelong learning system unable to adapt during execution is a literal oxymoron, and ultimately untenable.

The existence of an alternative computational paradigm has been proven mathematically1.

Computational forms exhibiting different levels of computational power and attributes were demonstrated to exist on a continuum with Turing type computation at its low end and ‘Super- Turing’ (ST) computation at its upper end. Another facet of this continuum can be described in terms of the system’s need for preparing in advance for every eventuality. In fact, ST computation can only take place when provided with input from the environment. Super-Turing is thus not merely more powerful computationally but intrinsically different - dependent on factors not present in Turing computation including, but not limited to: ST computation is driven by external input, which changes the ‘program’; ST uses only as much precision as is necessary for a given computational step; ST may use random elements; ST may be asynchronous; ST may incorporate continuous values.

These ST elements parallel aspects of biological computation and were identified by Turing in the years following the 1936 introduction of his Universal Turing Machine as he sought to create an intelligent computational system2. This is not to say that Turing-type computation does not exist in biology. It has been suggested that as input changes the ST program, the ST program in turn changes, or, in effect, reprograms, a particular set of Turing-machine programs, which then execute the new instructions. This methodology may, in fact, describe neural processing, and it is a foundational concept in L2M theory.

Following are some properties driven by L2M theory:

Balanced networks: A possible realization of an L2M system is a plastic nodal network (PNN)

– as opposed to a fixed, homogeneous neural network. While plastic, the PNN must incorporate hard rules governing its operation, maintaining an equilibrium. If rules hold the PNN too strongly, it will not be plastic enough to learn, yet without some structure the PNN will not be able to operate at all.

1 H.T. Siegelmann, Neural Networks and Analog Computation: Beyond the Turing Limit, Birkhauser, Boston, December 1998, chapters 7, 12.

2 H. T. Siegelmann, “Turing on Super-Turing and Adaptivity”. J. Progress in Biophysics & Molecular Biology.

April (Sep) 2013, 113(1):117-26

Selective plasticity: The L2M system must also balance plasticity with knowledge acquisition and retention. A fixed network is unable to learn, while a network that changes too much cannot retain knowledge. Super-Turing computation has the intrinsic property of minimizing resource usage, that is, it uses only as much of any resource as is needed in any computational step. This same quality may limit the change to nodal areas not directly involved in a particular computation, leading to a balance between knowledge acquisition and memory retention. The PNN can only exist in the context of an overall system, which contains operational rules, and where safety is assured. This same feature can be seen in humans: No matter how we try, we will close our eyelids if an object approaches too close to our eye. Yet, like a biological organism, the overall system must achieve a balance, and while hardwired instincts are beneficial, if we were hardwired to drive a car, it would be disastrous any time our hardwired instructions for driving differed from actual circumstances.

Goal driven perception: Unlike online learning on stationary data, continual learning involves changing data distributions, concept drift, and noise. During operation, labels will most likely be unavailable. But even more challenging, information is not provided as a training sample and the system must actively chose its input for adaptation. Goal driven perception introduces the possibility of treating the same situation differently based on context and system goals - much like an animal in search of food or a mate, will focus less on other opportunities. This form of perception leads to data flows that integrate from top down with bottom up sensory data and internal recurrent processes. Goal driven perception helps prevent over-emphasizing less relevant inputs and focusing instead on critical inputs that require the system’s response or adaptation.

Adapting to novel situations: The system is driven by the need to adapt to novel situations or when new objectives are introduced. The system will have to know when to learn or present new behavior. This requires L2M systems to differ from current ML in that they must not exhibit catastrophic forgetting when learning new behaviors. There are various solutions to this issue that will be discussed in the biological inspiration subsection below.

Proposals should discuss the abovementioned considerations, along with others that are deemed pertinent, and include performance metrics for evaluating L2M capabilities. Also, plasticity presents a particular challenge for verification and assurance of safe and correct operation. L2M proposals should include considerations of this issue and offer ways to monitor and limit the system as it evolves.

B.2. Transferring Ideas from Biology

The L2M program considers inspiration from biological adaptive mechanisms as a supporting pillar of the project. Biological systems exhibit an impressive capacity to learn and adapt their structure and function throughout their lifespan, while retaining stability of core functions.

Taking advantage of adaptive mechanisms evolved through billions of years honing highly robust tissue-mediated computation will provide unique insights for building L2M solutions.

Technical Area 1 (TA1) of the L2M program will focus on functional system development, and take inspiration from known biological properties. Technical Area 2 (TA2) will involve computational neuroscientists and computational biologists in identifying and exploring biological mechanisms that underlie real-time adaptation for translation into novel algorithms.

The following list of possible features for a neuroscience-inspired PNN is provided as an example for TA1 proposers. It is neither meant to be exhaustive, nor to limit proposers. Rather, the aim is to illustrate L2M concepts and highlight examples of details proposers should consider in their L2M solutions:

a) The PNN will change architecture based on sensory input, in conjunction with existing memory, and system goals.

b) Memory and computation are part and parcel of each other; they are different aspects of the same nodal connections.

c) Sensory inputs (e.g., visual, auditory) are first processed in dedicated areas of the PNN, and later form associations with other inputs and memories; continuous adaptation is used to detect and recognize stimuli in natural environments; nodes are differentiated via algorithmic connection rules that suit them to different processing functions.

d) Biological neural networks have rich sets of signaling systems, which change neuronal characteristics adaptively on different scales (i.e., globally, regionally, locally, etc.), e.g., hormones, neuromodulators, calcium waves, neurotransmitters.

Following brain properties, a PNN may incorporate local and differential signaling (as in neural action potential frequencies), and only the subset of neighbors who can 'hear' at that frequency will be activated. This architecture produces an exponential number of possible circuits in what would otherwise be a single circuit – vastly enriching complexity, while limiting energy use.

e) Animals are born (initiated) with some rules of operations, i.e., instincts and motivations, some of which remain throughout life. Similarly in the PNN, behavior rules, in conjunction with learning rules, which define how nodes create, break, weaken and strengthen nodal connections may constitute fixed functions, parameterized by active memories, sensory input, and an external user. They may benefit from evolutionary development.

f) This PNN will be free of catastrophic forgetting due to properties, such as:

Differentiation prevents nodes from changing globally for any given input;

asynchronous update causes only some connections to change; resource minimization leads to local updates rather than global changes.

The human brain does not constitute the only biological system relevant to the L2M project. The project will also look to lower animal brains, and even organisms without brains due to reduced complexity, yet remarkable behaviors. Furthermore, the program may examine non-neural based adaptation in the environment for mechanisms that can aid in L2M development3. Cancer biology, for example, teaches us about the importance of citizenship among cells within a tissue, while organ regeneration in the salamander may suggest mechanisms for robust adaptation. Ideas are welcome from any natural system of any scale or type.

3 F. Baluska and M. Levin, “On Having No Head: Cognition throughout Biological Systems,” Front Psychol. 2016, 7: p. 902

B.3 Out of Scope

DARPA strongly encourages scientists from numerous fields, including computer science, mathematics, computational biology and neuroscience, engineering, and synthetic biology, to participate in this paradigm-changing research. Incremental changes to the state of the art in AI and neural networks, including homogeneous networks—e.g., deep networks, classical spiking networks—are out of scope. Likewise, theoretical methods that do not lead directly to practical computational improvement and studying biological adaptation or performing unrelated biological experiments without transferring the results to algorithmic mechanisms are also out of the scope of this AI-focused program.

C. Technical Areas

The L2M program comprises two Technical Areas (TAs), and the Government-led Center Group. TA1’s objective is to develop continual learning mechanisms operating in a unified system. TA1 performers will develop L2M machine learning software components, incorporating necessary features for learning from the environment, and integrate these into a unified system. TA2’s objective is to explore new ideas for lifelong learning tied to mechanisms found in nature and translate those mechanisms into algorithms. This effort may include biological experiments, or leverage existing biological data. Methods of mathematical and computational biology are welcome in both TAs.

DARPA believes that a broad interdisciplinary, collaborative approach, both within and between the two TA groups, will maximize the potential for effective cross-pollination of ideas. At the same time, DARPA recognizes that prospective performers may have excellent capabilities in only one TA. Therefore, applicants may propose to one or both TAs, however a separate proposal must be submitted for each TA. Different groups within the same institute may propose independently. All performers should be prepared to work closely with complementary groups in the program.

C.1 Lifelong Learning Machine Systems (Technical Area 1):

The objective of Technical Area 1 (TA1) is to develop Lifelong Learning Machine system(s) that display continual online learning with the following core capabilities:

1) Continual Learning – Systems must be capable of not only learning during training, but during execution / task performance.

2) Adaption to New Tasks & Circumstances – Systems must be capable of applying previously learned / acquired skills and/or knowledge to novel tasks and circumstances in order to complete or navigate them without outside assistance.

Furthermore, systems may not eliminate previous knowledge when learning a new task.

3) Goal Driven Perception – Understanding input signals from the focus of the mission.

4) Selective Plasticity – Systems must be capable of balancing stability and plasticity.

5) Monitoring and Safety -- Systems must include hard coded rules, not accessible by the PNN, that among other functions, sets limits on system’s behavior (“Safety Failsafe”). L2M applications must allow users to monitor and intervene as needed.

The above list can be complemented by other features. TA1 performers should integrate core capabilities into a unified system and demonstrate its capabilities on test problems.

TA1 proposers should include the following topics among those discussed in their proposal.

a) Research and integration plan – present a roadmap for the development of L2M components that achieve project objectives. Define an L2M system’s architecture.

How do you plan to develop these mechanisms? Is the mechanism bio-inspired? If so, describe the source, its functionality, and application in terms of L2M. Which of them will you start with? Will you develop them in parallel? In a pipeline? Describe how your proposed L2M technologies will advance the capabilities of the chosen application as compared to state-of-the-art technologies. Include TA1 milestones outlined in Section E as well as the milestones specific to your project.

b) Test problem – proposers will define one or more demonstration applications of their choosing (see Section D for guidance).

c) Show broad applicability – provide a plan for demonstrating that solutions are general rather than limited to a single domain, such as a secondary application area or convincing analytics.

d) Validation, benchmarking, and evaluation – propose metrics and benchmarking to demonstrate that the L2M system will have significant new capabilities (e.g., ability to use unanticipated inputs and task contexts).

TA1 teams will define and generate their own Phase 1 demonstrations either in software-only environments or actual physical systems (e.g., robots) with the requirement that performers who choose to work with non-software systems must have existing expertise with such systems.

Funding will not be provided for performers to spend significant effort in learning how to operate or maintain physical systems. The L2M Center Group will support TA1 performers in defining additional demonstrations (see Section D). All research in L2M, including the evaluation on the common test problems, must be unclassified and must be suitable for publication in regular scientific venues.

C.2 Physical Principles of Machine Learning from Nature (Technical Area 2):

Technical Area 2 (TA2) will identify biological learning mechanisms to inform the development of new algorithms to address one or more L2M features (continual learning, adaptations to new tasks and circumstances, goal driven perception, selective plasticity, monitoring and safety).

Examples include:

1) Stability – how does the biological network retain information while changing?

2) Signaling – Will differential signaling and/or multiple adaptable types of nodes elevate the level of complexity in a useful manner?

3) Energy usage – how does the natural lifelong learning system adapt without using significant energy?

4) New skills – what mechanisms will the biological system use to understand what new skills are required?

5) Robustness and safety – what does safety mean in the candidate natural system and how is it supported?

In TA2, DARPA is interested in both biological and computational research and anticipates TA2 teams to comprise collaborations of researchers from the natural sciences (biology, neuroscience, physics, etc.) and mathematics, computer science, engineering, etc. TA2 teams will be required to demonstrate and analyze the value of their approaches for addressing one or more L2M core capabilities. TA2 proposals should offer an effective demonstration.

TA2 proposers should include the following topics among those discussed in their proposal:

a) Machine learning capability – describe the algorithmic capability for lifelong learning you will develop based on biological principles. What will it enable? How will you demonstrate this capability in your final algorithm?

b) Natural principle/mechanisms – describe the biological system/mechanism that addresses a chosen capability. What is known about the system? What more must be known to enable new approaches to machine learning? What data will you use to inform your algorithms? Does it exist already, or will you need to create it?

c) Research and integration plan – present a roadmap for the study of these principles and their incorporation into machine learning algorithms. Include TA2 milestones outlined in Section E as well as the milestones specific to your project. At a minimum, include annual milestones to track progress.

d) Validation and evaluation – what metrics will you use to assess the benefit of incorporating your suggested mechanisms?

C.3 Center Group

The intent of the L2M Center Group is to encourage cross-pollination of emerging approaches to the most difficult challenges in developing continuous machine learning across the entire L2M program, to identify new applications enabled by L2M advances, and to aid performers in accomplishing their tasks. The Center Group will be the focus of collaborations between performers and the Government and share expertise in areas of biology and computation pertaining to L2M goals and objectives. We intend that the Center Group will function as a program-level intelligence, developing collective understanding through our shared experience and insights. That performers might expose propriety data within the Center Group is neither expected nor desired. The Government team will lead the group. The L2M Center Group will include one technical representative from each performer team working in concert with the Government and will meet regularly throughout the program. Proposers should include cost information for one representative of each team to attend the Center Group meetings that will be held throughout the program (see item F, “Deliverables” below).

D. System Development Environments and Evaluation (TA1 only)

The overall challenge is to develop a system that is capable of making adjustments based on multiple sensory inputs during run time to conditions it has not encountered before and which it was not trained on. Demonstrating such capability is the key indicator of success in the L2M program. Performers will need to employ an environment or a dataset to develop and test their system. The environment or dataset will provide the necessary elements for system development and demonstration of the system’s ability to learn during execution time and to incorporate previously learned information in approaching novel situations. The performer’s own environment/dataset will be used for evaluation of progress during Phase 1 and will continue into Phase 2 with increasing system capabilities.

Proposers should describe their own tasks in systems that need to evolve. Proposed development / test environments should adequately demonstrate all the core L2M capability objectives listed in C.1. Two suggested settings are Autonomy and Identification/Prediction. In the Autonomy setting, the system takes actions to explore and alter its environment to accomplish tasks and must learn to adapt to significant changes in both task and environment. In the Identification/Prediction setting, the system learns to recognize and predict patterns and behaviors from multi-modal data with few or no explicit labels and must handle challenges such as non-stationarity and sensor degradation (e.g. modalities that are noisy or not always available). It is suggested that proposers describe their own problem within one of these two settings. Performers should develop L2M solutions for broad applicability, rather than limited point solutions.

The Center Group will provide additional scenarios. Common test problems developed by the Center Group will be gradually introduced in both Phase 1 and Phase 2 to validate TA1 core capabilities. These test problems will enable performers to evaluate their solutions against truly unforeseen dynamic changes.

Figure 2. System Evaluation – Proposer Identified and Center Group tasks

Common Test Problems Via the L2M Center Group, the Government will work closely with performers to identify common test problems, datasets, and environments that can be used for demonstrations and evaluation. This will allow performers to try alternative use cases and avoid excessively tailored solutions. The exact nature of the required efforts will depend on the specific technical approaches proposed by the L2M performers. The process of generating these common test

Phase 1 Year 1 Year 2

Phase 2 Year 3 Year 4

Performer Identified Tasks

Center Group’s Common Tasks problems will occur during the first half of the program, with implementation likely taking place in early Phase 2 via task-add contract modification/s at DARPA discretion.

E. Program Schedule and Milestones

The L2M program is a four-year program divided into two, two-year phases. Phase 1 (base) constitutes the concept development phase, in which novel system approaches and algorithms will be pursued and biological mechanisms not previously applied as machine learning methods will be explored. During Phase 1, metrics will be developed and evaluations of algorithms will be performed. Algorithms will continue to be developed and matured in Phase 2 according to the performer’s metrics. Phase 2 (option) demonstration applications will be completed and evaluated, including methods for monitoring and safety. In addition to the performer’s own application(s), common tests generated by the Center Group will be introduced starting at month

18. Performers will use the common datasets to show at least one of the TA1 core capabilities in Phase 1 and the full L2M capabilities in Phase 2.

E.1 TA1 milestones

Component Phase 1 – Proof of Concept Final – System Demonstration

Continual Learning

Show capability to improve performance while operating in the absence of labeled training data as well as the capability to adapt to increasing data variation over time.

Demonstrate a system with continual operation that shows robustness to noisy and spurious (surprise) data.

Adapting to New Tasks & Circumstances

Show capability to adapt to new tasks and circumstances without starting from scratch. Training data can be used.

Develop framework and demonstrate capability to switch tasks and behaviors, building upon previous ones, in the absence of labeled training data.

Goal driven perception

Show methodology for top down feedback that impacts lower level processing. Provide plans for incorporating the methodology into application.

Incorporate top down feedback where high-level objectives impact network configuration and data reduction into demonstration application.

Selective Plasticity

Show capability to selectively adapt to new situations within computing resource constraints.

Suggest theory and algorithmic method to adapt efficiently within the L2M framework.

Monitoring and Safety

Show capability to monitor system while it evolves. Suggest principles for security and provisions for accepting user intervention.

Suggest means to increase security and safety, so that system will only improve on initial safety conditions.

Table 1 – L2M Capability Components and Demonstrations

Milestones will be suggested by the individual groups. The program will progress as follows:

1) Phase 1, month 6: Identify individual lifelong learning approaches that will be under development for all core capability components in Phase 1; describe chief test problem(s) and corresponding metrics.

2) Phase 1, in reports: Provide updated analysis, and identify advancements in L2M systems, networks, and algorithms with respect to defined metrics.

3) Phase 1, month 18: Center Group provides common test data to performers for demonstrating core capabilities.

4) Phase 1, month 21: Provide initial Phase 2 plans, e.g., improvements and integration; introduce in detail the environment/dataset for upcoming demonstration. A common test problem is selected.

5) Phase 1 Final, month 24: Demonstration of all system, network, and algorithm capabilities; software library framework showing initial API, system requirements, and usage. One L2M core capability demonstrated using common test data that was provided in month 18. Plan that indicates choice and approach for demonstrating common task which will include all the core capabilities.

6) Phase 2, month 30: Provide plans for improved systems, networks, and algorithm capabilities; provide design for System Monitor application interface and system security.

7) Phase 2, in reports: Provide updated analysis and identify advancements in L2M systems, networks, and algorithms.

8) Phase 2, month 45: Demonstration of all L2M systems, networks, and algorithms, analysis, common test problem solutions, and capabilities; technology transfer plan for continued use of technology.

9) Completion of Phase 2, month 48: Final report, including detailed documentation of all critical algorithms.

E2. TA2 Milestones

The objective of TA2 is the development of new approaches for advancing machine learning that also inform biology/neuroscience. Proposers should pursue multiple approaches in both Phase 1 and 2.

1) Phase 1, month 6: Identification of current biological knowledge to be targeted for systems, networks, and algorithm development, including approach for experiments, expected data, and plan for converting biological mechanism(s) into L2M solutions.

2) Phase 1, in reports: Describe new data, systems, networks, and algorithms, software, and demonstrations, including metrics.

3) Phase 1, month 21: Initial Phase 2 plans, e.g., new mechanisms, improvements in knowledge of current ones or plans for their transfer, algorithms, and integration;

introduce in detail the environment/dataset for upcoming demonstration.

4) Phase 1 Final, month 24: Demonstration of all capabilities; software library framework showing initial API where relevant.

5) Phase 2, month 30: Plans for implementing improved knowledge of using biological mechanisms investigated in Phase 1, perhaps investigating additional biological systems; report on issues of energy and safety in the systems investigated.

6) Phase 2, in reports: Describe new data, systems, networks, and algorithms, software, and demonstrations.

7) Phase2, month 45: Demonstration of all mechanisms, computational models, L2M systems, networks, and algorithms, analysis, and capabilities; and if relevant, a technology transfer plan.

8) Phase 2 Final, month 48: Final report, including detailed documentation of all critical algorithms.

F. Deliverables

Proposals must include a detailed schedule of deliverables. Performers are especially encouraged to define appropriate deliverables that provide benchmarks and indicate progress against the highest risk elements of their approach. These should include, at a minimum, the following deliverables, as applicable:

Monthly Coordination Reports / Teleconferences to present progress made (including studies, analyses, and metrics), plans for the upcoming month, and any issues requiring the attention of DARPA

Monthly Financial Reports detailing incurred and projected resource expenditures and travel

Quarterly technical reports providing detailed discussion of the approach taken and progress made (including studies, analyses, and metrics) as well as plans for the next reporting period. Semi-annual and annual reports will take a broader scope. Annual reports will be self-contained documents in the style of academic papers

Any publications covering work funded by L2M.

Plans, roadmaps, detailed documentation of all critical algorithms, and demonstrations as specified in Section E.

Report on monitoring and safety in L2M systems, especially addressing questions pertaining to safety and assurance in lifelong learning systems Center Group meetings, throughout: A representative from each performer team must attend ten Phase 1 and five Phase 2 three-day meetings of the L2M Center Group in or near DARPA in Arlington, VA

Final technical report at the completion of each phase, summarizing the effort conducted and providing lessons learned during the development of the technology

The Government team will hold four program reviews each year with individual performers. For planning purposes, assume that two of these reviews will be in or near DARPA in Arlington, VA. The other two reviews will be hosted at the performer or team member facility or via remote conferencing. Typically, two of these will coincide with the program-wide program review.

There will be two program-wide review meetings each year, for all performers to attend together, which will include the Phase 1 and Phase 2 kick-off meetings as well as a final meeting each year. For planning purposes, assume the meetings will cover three days.

Meeting Summary Table

Timeline Meeting Purpose

Monthly Progress and coordination teleconference

Quarterly

Program review with individual performers, two of which will be held at or near DARPA and two of which will be held either at the performer or team site or remotely

Semi-annual Three-day program-wide review with all performers, including phase kick-offs and end-of-year close-out meetings.

Phase 1 Center Group 10 three-day meetings at or near DARPA

Phase 2 Center Group 5 three-day meetings at or near DARPA

Phase 1 Month 24 Demonstration

Phase 2 Month 45 Demonstration

Deliverable Summary Table

Timeline Deliverable

Monthly Technical progress and financial report slides, including studies, analyses, and metrics

Quarterly Increasingly detailed, cumulative technical progress report, including studies, analyses, and metrics

Semi-annual

Increasingly detailed, cumulative technical progress report, including TA1, Phase 1, month 6: identification of approaches to implement all core capabilities; description of Phase 1 development and test problems and metrics

TA2, Phase 1, month 6: identification of biological mechanism to target for algorithm development, expected data, and plan for implementation

Annual Detailed, self-contained cumulative technical progress report in the style of academic paper

Phase 1 Month 21 Phase 2 application plans

Phase 1 Month 24 TA1: Demonstration, software library framework

Phase 1 Month 36 TA1: Design review of common test application

Phase 2 Month 45 Demonstration. For TA1, also transfer plan; monitoring and safety report

Phase 2 Month 48

Final report, including detailed documentation of all critical algorithms for developed software

II. Award Information

A. General Award Information

Multiple awards are anticipated. The amount of resources made available under this BAA will depend on the quality of the proposals received and the availability of funds.

The Government reserves the right to select for negotiation all, some, one, or none of the proposals received in response to this solicitation, and to make awards without discussions with proposers. The Government also reserves the right to conduct discussions if it is later determined to be necessary. If warranted, portions of resulting awards may be segregated into pre-priced options. Additionally, DARPA reserves the right to accept proposals in their entirety or to select only portions of proposals for award. In the event that DARPA desires to award only portions of a proposal, negotiations may be opened with that proposer. The Government reserves the right to fund proposals in phases with options for continued work at the end of one or more of the phases, as applicable.

Awards under this BAA will be made to proposers on the basis of the evaluation criteria listed below (see section labeled “Application Review Information,” Sec. V.), and program balance to provide overall value to the Government. The Government reserves the right to request any additional, necessary documentation once it makes the award instrument determination. Such additional information may include but is not limited to Representations and Certifications (see Section VI.B.2., “Representations and Certifications”). The Government reserves the right to remove proposers from award consideration should the parties fail to reach agreement on award terms, conditions and cost/price within a reasonable time or the proposer fails to timely provide requested additional information. Proposals identified for negotiation may result in a procurement contract, grant, cooperative agreement, or other transaction, depending upon the nature of the work proposed, the required degree of interaction between parties, whether or not the research is classified as Fundamental Research, and other factors.

Proposers looking for innovative, commercial-like contractual arrangements are encouraged to consider requesting Other Transactions. To understand the flexibility and options associated with Other Transactions, consult www.darpa.mil/work-with-us/contract-management#OtherTransactions.

In all cases, the Government contracting officer shall have sole discretion to select award instrument type, regardless of instrument type proposed, and to negotiate all instrument terms and conditions with selectees.

B. Fundamental Research

It is DoD policy that the publication of products of fundamental research will remain unrestricted to the maximum extent possible. National Security Decision Directive (NSDD) 189 defines fundamental research as follows:

‘Fundamental research’ means basic and applied research in science and engineering, the results of which ordinarily are published and shared broadly within the scientific http://www.darpa.mil/work-with-us/contract-management#OtherTransactions community, as distinguished from proprietary research and from industrial development, design, production, and product utilization, the results of which ordinarily are restricted for proprietary or national security reasons.

As of the date of publication of this BAA, the Government expects that program goals as described herein can be met by proposers intending to perform fundamental research and does not anticipate applying publication restrictions of any kind to individual awards that result from this BAA.

Notwithstanding the above, the Government shall have sole discretion to select award instrument type and to negotiate all instrument terms and conditions with selectees.

III. Eligibility Information

All responsible sources capable of satisfying the Government's needs may submit a proposal that shall be considered by DARPA.

A. Eligible Applicants

1. Federally Funded Research and Development Centers (FFRDCs) and Government Entities

a) FFRDCs

FFRDCs are subject to applicable direct competition limitations and cannot propose to this BAA in any capacity unless they meet the following conditions: (1) FFRDCs must clearly demonstrate that the proposed work is not otherwise available from the private sector. (2) FFRDCs must provide a letter on official letterhead from their sponsoring organization citing the specific authority establishing their eligibility to propose to Government solicitations and compete with industry, and their compliance with the associated FFRDC sponsor agreement’s terms and conditions. This information is required for FFRDCs proposing to be awardees or subawardees.

b) Government Entities

Government Entities (e.g., Government/National laboratories, military educational institutions, etc.) are subject to applicable direct competition limitations. Government entities must clearly demonstrate that the work is not otherwise available from the private sector and provide written documentation citing the specific statutory authority and contractual authority, if relevant, establishing their ability to propose to Government solicitations.

c) Authority and Eligibility

At the present time, DARPA does not consider 15 U.S.C. § 3710a to be sufficient legal authority to show eligibility. While 10 U.S.C.§ 2539b may be the appropriate statutory starting point for some entities, specific supporting regulatory guidance, together with evidence of agency approval, will still be required to fully establish eligibility. DARPA will consider FFRDC and Government entity eligibility submissions on a case-by-case basis; however, the burden to prove eligibility for all team members rests solely with the proposer.

2. Non-U.S. Organizations and/or Individuals

Non-U.S. organizations and/or individuals may participate to the extent that such participants comply with any necessary nondisclosure agreements, security regulations, export control laws, and other governing statutes applicable under the circumstances.

B. Organizational Conflicts of Interest

FAR 9.5 Requirements In accordance with FAR 9.5, proposers are required to identify and disclose all facts relevant to potential OCIs involving the proposer’s organization and any proposed team member (subawardee, consultant). Under this Section, the proposer is responsible for providing this disclosure with each proposal submitted to the BAA. The disclosure must include the proposer’s, and as applicable, proposed team member’s OCI mitigation plan. The OCI mitigation plan must include a description of the actions the proposer has taken, or intends to take, to prevent the existence of conflicting roles that might bias the proposer’s judgment and to prevent the proposer from having unfair competitive advantage. The OCI mitigation plan will specifically discuss the disclosed OCI in the context of each of the OCI limitations outlined in FAR 9.505-1 through FAR 9.505-4.

Agency Supplemental OCI Policy In addition, DARPA has a supplemental OCI policy that prohibits contractors/performers from concurrently providing Scientific Engineering Technical Assistance (SETA), Advisory and Assistance Services (A&AS) or similar support services and being a technical performer.

Therefore, as part of the FAR 9.5 disclosure requirement above, a proposer must affirm whether the proposer or any proposed team member (subawardee, consultant) is providing SETA, A&AS, or similar support to any DARPA office(s) under: (a) a current award or subaward; or (b) a past award or subaward that ended within one calendar year prior to the proposal’s submission date.

If SETA, A&AS, or similar support is being or was provided to any DARPA office(s), the proposal must include:

The name of the DARPA office receiving the support;

The prime contract number;

Identification of proposed team member (subawardee, consultant) providing the support; and An OCI mitigation plan in accordance with FAR 9.5.

Government Procedures In accordance with FAR 9.503, 9.504 and 9.506, the Government will evaluate OCI mitigation plans to avoid, neutralize or mitigate potential OCI issues before award and to determine whether it is in the Government’s interest to grant a waiver. The Government will only evaluate OCI mitigation plans for proposals that are determined selectable under the BAA evaluation criteria and funding availability.

The Government may require proposers to provide additional information to assist the Government in evaluating the proposer’s OCI mitigation plan.

If the Government determines that a proposer failed to fully disclose an OCI; or failed to provide the affirmation of DARPA support as described above; or failed to reasonably provide additional information requested by the Government to assist in evaluating the proposer’s OCI mitigation plan, the Government may reject the proposal and withdraw it from consideration for award.

C. Cost Sharing/Matching

Cost sharing is not required; however, it will be carefully considered where there is an applicable statutory condition relating to the selected funding instrument. Cost sharing is encouraged where there is a reasonable probability of a potential commercial application related to the proposed research and development effort.

For more…