HR001117S0045.pdf

Broad Agency Announcement Assured Autonomy

HR001117S0045

August 18, 2017

Defense Advanced Research Projects Agency Information Innovation Office 675 North Randolph Street Arlington, VA 22203-2114

HR001117S0045 ASSURED AUTONOMY 2

Table of Contents

I. Funding Opportunity Description

A. Introduction

B. Program Description

C. Program Structure

D. Technical Areas

E. Schedule

F. Milestones

G. Deliverables

H. Intellectual Property

II. Award Information

A. Awards

B. Fundamental Research

C. Disclosure of Information and Compliance with Safeguarding Covered Defense Information Controls

III. Eligibility Information

A. Eligible Applicants

B. Organizational Conflicts of Interest

C. Cost Sharing/Matching

D. Other Eligibility Requirements

IV. Application and Submission Information

A. Address to Request Application Package

B. Content and Form of Application Submission

C. Submission Dates and Times

D. Funding Restrictions

E. Other Submission Requirements

V. Application Review Information

A. Evaluation Criteria

B. Review and Selection Process

VI. Award Administration Information

A. Selection Notices

B. Administrative and National Policy Requirements

C. Reporting

VII. Agency Contacts

HR001117S0045 ASSURED AUTONOMY 3

VIII. Other Information

A. Frequently Asked Questions (FAQs)

B. Proposers Day

C. Submission Checklist

D. Associate Contractor Agreement Clause (ACA)

HR001117S0045 ASSURED AUTONOMY 4

PART I: OVERVIEW INFORMATION

Federal Agency Name: Defense Advanced Research Projects Agency (DARPA), Information Innovation Office (I2O)

Funding Opportunity Title: Assured Autonomy

Announcement Type: Initial Announcement

Funding Opportunity Number: HR001117S0045

Catalog of Federal Domestic Assistance Numbers (CFDA): Not Applicable

Dates o Posting Date: August 18, 2017 o Proposers Day: August 15, 2017 o Abstract Due Date: September 6, 2017, 12:00 noon (ET) o Proposal Due Date: October 19, 2017, 12:00 noon (ET) o BAA Closing Date: October 19, 2017, 12:00 noon (ET)

Anticipated Individual Awards: DARPA anticipates multiple awards in technical areas (TAs) 1, 2, 3, and 4.

Total Funding Available for Award:

Types of Instruments that May be Awarded: Procurement contracts or Other Transactions

Agency Contacts o Technical POC: Dr. Sandeep Neema, Program Manager, DARPA/I2O o BAA Email: AssuredAutonomy@darpa.mil o BAA Mailing Address:

DARPA/I2O

ATTN: HR001117S0045

675 North Randolph Street Arlington, VA 22203-2114 o I2O Solicitation Website: http://www.darpa.mil/work-with-us/opportunities mailto:AssuredAutonomy@darpa.mil http://www.darpa.mil/work-with-us/opportunities

HR001117S0045 ASSURED AUTONOMY 5

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

The DARPA Information Innovation Office (I2O) is soliciting innovative research proposals to substantially improve the design, integration and verification/testing technology for continual assurance of autonomous learning-enabled cyber-physical systems to guarantee their safety and performance. Proposed research should investigate innovative approaches that enable revolutionary advances in science, devices, or systems. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice.

This Broad Agency Announcement (BAA) is being issued, and any resultant selection will be made, using procedures under Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016.

Any negotiations and/or awards will use procedures under FAR 15.4. Proposals received as a result of this BAA shall be evaluated in accordance with evaluation criteria specified herein through a scientific review process.

DARPA BAAs are posted on the Federal Business Opportunities (FBO) website (https://www.fbo.gov/).

The following information is for those wishing to respond to this BAA.

A. Introduction

Autonomous systems are increasingly critical to several current and future Department of Defense (DoD) mission needs. The U.S. Army Robotics and Autonomous Systems (RAS) strategy report for 2015-2040 identifies a range of capability objectives – including enhanced situational awareness, cognitive workload reduction, force protection, cyber defense, logistics, etc. – that rely on autonomous systems and higher levels of autonomy. In this BAA, autonomy is referring to a system’s ability to accomplish goals independently, or with minimal supervision from human operators in environments that are complex and unpredictable.

Tremendous advances have been made in the last decade in constructing autonomy-capable Cyber Physical Systems (CPS), as evidenced by the proliferation of a variety of unmanned systems – air, ground, sea and undersea vehicles. These advances have been driven by innovations in several areas – sensor and actuator technologies, computing technologies, control theory, design methods and tools, modeling and simulation technologies, among others. In spite of these advances, deployment and broader adoption of such systems in safety-critical DoD applications remains challenging and controversial.

Several factors impede the deployment and adoption of autonomous systems:

1. In the absence of an adequately high level of autonomy that can be relied upon, substantial operator involvement is required, which not only severely limits operational gains, but creates significant new challenges in the areas of human-machine interaction and mixed initiative control.

2. Achieving higher levels of autonomy in uncertain, unstructured, and dynamic environments, on the other hand, increasingly involves data-driven machine learning https://www.fbo.gov/

HR001117S0045 ASSURED AUTONOMY 6

techniques with many open systems science and systems engineering challenges.

3. Machine learning techniques widely used today are inherently unpredictable and lack the necessary mathematical framework to provide guarantees on correctness, while DoD applications that depend on safe and correct operation for mission success require predictable behavior and strong assurance.

The 2016 Defense Science Board (DSB) Report on Autonomy1 strongly emphasizes the need for trustworthiness and trust. Assuring that systems will operate safely and perform as expected is integral to trust, which is core to DoD’s success in adoption of autonomy. While an autonomous system must be designed to operate in a trustworthy fashion with respect to its expected missions, it must also be designed so that operators can determine whether, once it has been deployed, it is operating reliably and within its envelope of competence — and, if not, that appropriate action can be taken. Establishing trustworthiness at design time and providing adequate capabilities so that inevitable variations in operational trustworthiness can be assessed and dealt with at operation time is essential.

Historically, assurance has been approached through design processes following rigorous safety standards2 in development, and demonstrating compliance through system testing. However, these standards have been developed primarily for human-in-the-loop systems, and are bounded in scope, not extending to advanced levels of autonomy where system behavior depends on its memory of received stimuli. Current assurance approaches are predicated on the assumption that once the system is deployed, it does not learn and evolve.

One approach to assurance of autonomous systems that has recently garnered attention, particularly in the context of self-driving vehicles, is based on the idea of “equivalent levels of safety,” i.e., the autonomous system must be at least as safe as a comparable human-in-the-loop system that it supplants. The approach entails a statistical argument comparing known rates of safety incidents of manned systems (e.g., number of accidents per thousand miles driven in the context of self-driving cars), and conducting physical trials to determine the corresponding incident rate for autonomous systems. Studies3 and analyses indicate, however, that assuring safety of autonomous systems in this manner, by statistical testing alone, is very challenging, requiring a large number of physical trials potentially consuming much time. Arguably, simulation techniques can supplement and reduce some of the physical trials, but they provide little confidence in terms of sufficiency of coverage, particularly with respect to low-probability/high-consequence events.

In contrast to prescriptive, process-oriented standards for safety and assurance, a goal-oriented approach is better suited to systems that learn and evolve, and encounter operational variations.

In the goal-oriented paradigm, assurance cases4 are developed to provide a systematic and structured argument with both design and operation time evidence that a system satisfies explicitly stated assurance and safety goals. The approach results in a formal and potentially machine-analyzable, quantitative measure of assurance that can also evolve with the system.

1 Report of the Defense Science Board Summer Study on Autonomy 2016:

http://www.acq.osd.mil/dsb/reports/2010s/DSBSS15.pdf.

2 e.g. IEC 61508, ISO 26262, DO 178B/C.

3 Kalra N., Paddock S., “Driving to Safety – How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability?,”, Report, RAND Corporation, April 2016.

4 Rushby J., “Interpretation and Evaluation of Assurance Cases,” Technical Report, SRI International, July 2015.

http://www.acq.osd.mil/dsb/reports/2010s/DSBSS15.pdf

HR001117S0045 ASSURED AUTONOMY 7

B. Program Description

The goal of the Assured Autonomy program is to create technology for continual assurance of Learning-Enabled, Cyber Physical Systems (LE-CPSs). Continual assurance is defined as an assurance of the safety and functional correctness of the system provided provisionally at design time, and continually monitored, updated, and evaluated at operation-time as the system and its environment evolves. An LE-CPS is defined as a system composed of one or more Learning-enabled Components (LECs). A LEC is a component whose behavior is driven by “background knowledge” acquired and updated through a “learning process”, while operating in a dynamic and unstructured environment. This definition generalizes and admits a variety of popular machine learning approaches and algorithms (e.g., supervisory learning for training classifiers, reinforcement learning for developing control policies, algorithms for learning system dynamics). The generalization is intentional to promote abstractions and tools that can be applied to different types and applications of data-driven machine learning algorithms in CPSs to enhance their autonomy.

In order to concretize and ground the research objectives, the program will prioritize challenge problems in the militarily relevant autonomous vehicle space. However, it is anticipated that the tools, toolchains, and algorithms created will be relevant to other LE-CPSs. The resulting technology from the program will be in the form of a set of publicly available tools integrated into LE-CPS design toolchains that will be made widely available for use in commercial and defense sectors.

The program seeks revolutionary breakthroughs in three closely-coupled technical areas as depicted in the notional architecture in Figure 1 below.

Figure 1: Assured Autonomy Program Architecture and Technical Areas 1-3

HR001117S0045 ASSURED AUTONOMY 8

The subsequent description provides a brief overview of the notional architecture; further details regarding the technical areas (TA) are provided in later sections.

(TA1) Design for Assurance requires foundational innovations in technologies for design and verification of LE-CPS. The inclusion of learning in CPSs poses unique challenges for design and software engineering, succinctly articulated as “hidden technical debt” in machine learning systems5. For safety critical CPSs, the design technologies not only should generate an implementation for the system, but should also produce evidence regarding safety and correctness of the design. The program seeks novel multi-domain modeling formalisms for representation of LE-CPSs. These representations will drive novel approaches for formal verification, simulation, and testing to generate evidence for correctness. A subset of evidence may be conditional (depicted as E’ in Figure 1), and subject to assumptions made about the operating environment or about the state of the “background knowledge” of the LEC. The parameters underlying all conditional evidence will need to be explicated and monitored at operation-time.

(TA2) Assurance Monitoring and Control requires breakthroughs in operation-time techniques for assured and safe operation of LE-CPS. The program seeks to create safety-aware learning, innovative techniques that render the learning algorithms inherently safe by incorporating safety constraints in the learning process, while meeting learning objectives. The program also seeks to create assurance guards and monitors, which are low-overhead and scalable approaches to monitor critical safety properties and the parameters for conditional evidence. The outputs of assurance monitors will need to update conditional evidence, and augment and evolve the assurance case.

(TA3) Dynamic Assurance integrates design time assurance with operation time assurance, and requires innovations in assurance case technologies to deliver a formal and quantitative measure of assurance that is dynamic and evolves with the system. New approaches to assurance will formalize and admit conditional evidence at design time, and synthesize computable structures that can be updated and evaluated during operation time.

(TA4) Integration and Experimentation Platform provides challenge problems for evaluation and demonstration of the assurance technologies developed in the program. The challenge problems are expected to be scenarios or mission vignettes of increasing complexity that the platform must execute autonomously. As an example of a platform, consider an autonomous surface ship that is capable of autonomous navigation in open seas. A platform provider, with such a platform, would augment the platform with learning technologies, if it is not already learning-enabled, and develop complex scenarios stressing online and offline learning in complex evolving environments as challenge problems. The assurance technology developers will develop, demonstrate, and validate a dynamic assurance case for the challenge problems.

5 Sculley et al, “Hidden Technical Debt in Machine Learning Systems,” https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf.

HR001117S0045 ASSURED AUTONOMY 9

C. Program Structure

The Assured Autonomy program has four TAs, depicted in Figure 2 below.

Figure 2: Assured Autonomy Technical Areas

Proposers may address any of the four TAs, which are discussed further in Section I.D. To ensure independence and prevent conflicts of interest, proposers selected for TA4 will not be selected for award as a performer (prime or subcontractors) on TA1, TA2, or TA3. The decision as to which technical area(s), if any, to consider for award is at the discretion of the Government.

See Section III.D.1 for additional information.

The anticipated interactions between the TAs, notionally depicted in Figure 2, are as follows.

The assurance technologies developed by TA1, TA2, and TA3 performers will be evaluated against the challenge problems developed by TA4 performers for their respective autonomous platforms. The goal of the program is not to develop new autonomous platforms, but rather, augment the platform with learning technologies, and develop assurance for the learning-enabled autonomous platform. Correspondingly, DARPA expects that the TA4 performers will propose a mature, militarily-relevant autonomous platform – examples of such platforms may include autonomous surface ships, underwater vehicles, ground or air vehicles. DARPA also expects that the TA4 performers will provide access to high-fidelity simulators or surrogate platforms to facilitate technology development by TA1-3

The TA4 performers will provide platform definitions – architecture and behavior descriptions, component and system models, goals and safety requirements; and challenge problem scenario descriptions to the TA1-3 performers. The TA3 performer will formulate an assurance case for

HR001117S0045 ASSURED AUTONOMY 10

the platform safety and integrate evidence from TA1 and TA2 performers. TA2 performers will develop operational components, including safe learning algorithms and assurance monitors that will need to be integrated on the platform by the TA4 performer. DARPA anticipates technology development teams consisting of a TA3 performer, and one or more TA1 and TA2 performer, to work on the TA4 challenge problems.

Program success will require a close and continued collaboration from a range of fields, including but not limited to modeling and system design, formal verification, simulation-based testing, machine learning, safety-aware learning, program analysis, autonomy, control theory, and a multitude of application domains. Anticipated interactions across performers are as described above, however, all performers under the Assured Autonomy program will be expected to work cooperatively to develop, integrate, implement, and validate assurance autonomy capabilities. Therefore, all proposers should carefully review the expectations of all four TAs to fully understand and address the context of any TA(s) for which they will submit proposals. To facilitate the open exchange of information, performers shall have an Associate Contractor Agreement (ACA) clause (see Section VIII.D for sample) included in their award.

Performers should execute the ACAs prior to the program kick-off. This clause is intended to ensure appropriate coordination and integration of work by the performers, while maximizing commonality and preventing unnecessary duplication of effort.

The Assured Autonomy program is organized in three phases. Phase I will be eighteen (18) months and will consist of initial research and tool development. Phase II will be fifteen (15) months and will focus on technology enhancement and expansion of the scope of learning technologies targeted by tools in the program. Phase III will also be fifteen (15) months and will focus on technology maturation and capstone demonstration on experimentation platforms. Each phase will end with integrated demonstrations of the tools on the selected challenge problems, which will provide an opportunity to evaluate the progress made against the program objectives.

Assured Autonomy has several quantifiable objectives, among which are:

Increase scalability of design-time assurance – Verification methods for hybrid systems suffer from the curse of dimensionality, and the inclusion of learning components exacerbates the scalability challenge. Proposers should describe the baseline capability of their methods, in terms of the hybrid state-space and number and complexity of learning-enabled components, and how they plan to scale up by an order of magnitude.

Proposers considering multi-fidelity abstractions should also characterize the tradeoffs between fidelity of their modeling abstractions and scalability of the verification approach.

Reduce overhead of operation-time assurance – Proposers should describe the baseline overhead of the operation-time assurance monitoring techniques, and describe how they plan to minimize it to be below 10% of the nominal system resource utilization.

Scale up dynamic assurance – Proposers should describe the size and scale of dynamic assurance case that can be developed and dynamically evaluated with their tools.

Reduce trials to assurance – One of the key challenge of the program is to reduce the need for statistical testing by increasing the coverage using the assurance technologies developing in the program. Proposers should discuss how their approach can quantifiably reduce the need for statistical testing.

HR001117S0045 ASSURED AUTONOMY 11

DARPA is seeking an order of magnitude increase in the scale and complexity of the challenge problems across the phases, and correspondingly the assurance technologies developed in the program must scale up to address these challenges.

Table 1 below summarizes the key program objectives and the anticipated quantitative progression through the program phases for illustration purposes. Proposers should develop and describe quantitative metrics specific to their approach aligning with the objectives listed below.

Table 1: Assured Autonomy Objectives

TA Metric Phase 1 Phase 2 Phase 3

1 Scalability† 20 dimensions 1-2 LEC (low)

40 dimensions 2-4 LEC (mid)

100 dimensions 4-6 LEC (high)

2 Monitoring overhead 50% 30% 10%

3 Dynamic assurance 10 conditional evidence

100 conditional evidence

1000 conditional evidence

4 Reduced trials to assurance 0.1x 0.01x 0.001x

DARPA anticipates two TA4 platforms in the program to demonstrate domain agnosticism of the technology developed. The technology developers (TA1-3 performers) should plan to apply their technology to both the platforms, and the platform developers (TA4 performers) should plan to support multiple technology development (TA1-3) teams. To maximize flexibility for the Government, TA1-3 performer should include support for the 2nd platform as an option.

Similarly, TA4 performer should plan to support two (2) TA1-3 teams in their base effort, while provide options for supporting additional TA1-3 teams.

At program kick-off, DARPA will encourage TA1-3 teaming, for performers that are not already part of an integrated TA1-3 proposal. These TA1-3 teams, will work with each TA4 performer on addressing the platform-specific challenge problems, and producing an assured version of a learning-enabled autonomous system that will be demonstrated at the end of each phase. Thus, close collaboration between TA1-4 performers is expected. These teams will not be competitively evaluated and while there is no planned down-selection of either teams or individual performers the innovation in the approach, and technical progress demonstrated will determine continuation across phases.

The Assured Autonomy program will emphasize creating and leveraging open source technology. Intellectual property rights asserted by proposers are strongly encouraged to be aligned with open source regimes. See Section VI.B.1 for more details on intellectual property.

Proposals should specifically list anticipated technical and programmatic risks and describe associated mitigation strategies.

HR001117S0045 ASSURED AUTONOMY 12

D. Technical Areas

TA1: Design for Assurance

This technical area will focus on building tools for design, and verification of LE-CPS. The tools developed in this technical area should span all (cyber and physical) component types, including LEC, of the LE-CPS. TA1 performers will be responsible for developing the necessary formalisms and abstractions for representation of LECs, and novel approaches for formal and coverage-driven verification of LE-CPS. Strong proposals will focus on verification technologies that target relevant machine learning technologies such as (but not limited to) neural network-based classifiers or controllers based on reinforcement learning.

The tools developed in this technical area are expected to generate qualified evidence of safety and correctness of the LE-CPS. Evidence is qualified by explication of conditions and assumptions under which that evidence is valid. As an example, consider a high-level safety requirement such as “the autonomous ship shall not collide with a sail-boat crossing starboard of it that has the right of way”. The tools in TA1 will need to provide a proof that the property is satisfied, explicate all the conditions under which the proof is valid, and provide these conditions to TA2 to monitor. Strong proposals will be able to provide a graduated measure of validity (or confidence), that can be updated as monitored conditions evolve at operation time. TA3 will integrate this generated evidence into a comprehensive assurance case for the target system.

Research challenges in TA1 include, but are not limited to, development in the following areas:

Compositional architectures for learning-enabled systems that guarantee and preserve specified properties;

Formalisms, abstractions, and domain specific modeling languages for representation of learning-enabled components, systems and their dynamics;

Scalable methods addressing formal verification of safety and liveness properties of LE-

CPS;

Simulation approaches that drive the learning-enabled system to elicit unanticipated behaviors;

Approaches for maximizing test coverage of LE-CPS; and

Transformations for automated synthesis of assurance monitors.

Proposals for TA1 should address one or more of these challenges, clearly describing the innovations in the technical approach. Proposals can address other challenges not listed above, provided there is a strong justification for the centrality of the challenge to the achievement of TA1 goals. Proposals for a comprehensive solution to these areas are encouraged, though not required. Proposals must describe at least one example of a challenge problem (as a proxy for challenge problems that will be proposed by TA4 – consider the example in TA4 description for guidance), for which the proposed techniques can be applied effectively, clearly describing the metrics and capability milestones reached at the end of each phase. As the outcome of TA1 effort are tools to be integrated into LE-CPS design flows, TA1 proposals should address interfaces provided for integration.

HR001117S0045 ASSURED AUTONOMY 13

Performers in this area are expected to be part of aTA1-3 team. DARPA encourages, but does not require, integrated TA1-3 proposals. Proposers submitting a TA1 only proposal should anticipate joining a TA1-3 team post program kick-off, and describe their interface and plans for working with other TAs. Performers in this area will need to work closely with TA4 performers to learn about the target platforms, apply their techniques to the provided challenge problems, and consult on the application of their techniques to the target platform. TA1 performers must demonstrate their tools on appropriate portions of the challenge problems, and provide evidence that TA3 can use to construct a dynamic assurance case.

DARPA anticipates two TA4 platforms, and the integrated TA1-3 team will need to work on challenge problems posed by both platforms. To maximize flexibility for the Government, the proposers should plan to include support for one TA4 platform in the base effort, and include support for the second platform as an option.

TA2: Assurance Monitoring and Control

This technical area will develop techniques for operation-time monitoring and preservation of safety and correctness guarantees. The goal of this TA is to produce techniques that monitor and constrain the learning in such a way that it does not compromise safety, while optimizing performance with learning in a dynamic environment. This technical area has two related sub-areas:

1. Monitoring conditions and assumptions that impact the validity of the evidence for safety and correctness, and quantifying the impact (e.g., in terms of degraded confidence) in case of divergence; and

2. Making learning algorithms inherently safe.

The monitors will perform multiple roles, including but not limited to:

1. Enforcing hard limits for inviolable safety constraints;

2. Monitoring architectural constraints that are required to guarantee specific performance properties (such as control stability); and

3. Monitoring and assessing the robustness of learning-enabled components as their operating conditions diverge from the conditions of their training.

TA2 proposals should characterize proposed monitoring techniques with respect to their expressivity, coverage (i.e., type of properties that can be monitored), reactive vs. predictive, deterministic vs. probabilistic, and scalability (i.e., operation time overhead of the monitoring algorithm). Approaches to the inherently safe learning sub-area must address the challenge of combining safety analysis within specific learning algorithms, performance and metrics, and the scalability of the approach. Strong proposals will address learning algorithms that are most likely to be relevant to advanced LE-CPS.

Research challenges in this area include, but are not limited to, development in the following areas:

Techniques and algorithms for safety-aware learning;

Monitors for enforcement of hard safety constraints;

Monitors for enforcement of architectural constraints; and

HR001117S0045 ASSURED AUTONOMY 14

Monitors to detect data-distribution shifts, and qualifying the performance of learning algorithms as the operating environment diverges from training environment.

Proposals for TA2 should address one or more of these challenges, clearly describing the innovations in the technical approach. Proposals can address other challenges not listed above, provided there is a strong justification for the challenge being necessary to achieve the goals of the TA. Proposals that aim to provide a comprehensive solution to these areas are encouraged, though not required. Proposals must describe at least one example of a challenge problem (a proxy for challenge problems that will be proposed by TA4 – consider the example in TA4 section for guidance) for which the proposed algorithms can be applied effectively, clearly describing the metrics and capability milestones reached at the end of each Phase. The envisioned outcome of the TA2 effort are operation-time components that need to be integrated into the TA4 platforms. Proposals need to explain interfaces provided for integration.

Performers in this area are expected to be part of aTA1-3 team. DARPA encourages, but does not require, integrated TA1-3 proposals. Proposers submitting TA2 only proposal should anticipate joining a TA1-3 team post program kick-off, and describe their interface and plans for working with other TAs. Performers in this area will need to work closely with TA4 performers to learn about the target platforms, apply their techniques to the provided challenge problems, and consult on the application of their techniques to the target platform. TA2 will also require close coordination with TA1, as the conditions to be monitored and the assumptions to be validated will be provided by the TA1 performers. DARPA anticipates two TA4 platforms, and the integrated TA1-3 team will need to work on challenge problems posed by both platform. To maximize flexibility for the Government, the proposers should plan to include support for one

(1) TA4 platform in the base effort, and include support for the second platform as an option.

TA3: Dynamic Assurance

This technical area will focus on developing tools and algorithms for formal representation and dynamic evaluation of assurance cases. The program envisions a dynamic assurance case to be the framework that combines design time guarantees with run time monitoring to provide a continual assurance of safety and correctness of LE-CPS. TA3 needs to perform the following roles:

1. Provide domain expertise to formulate a robust assurance case for a learning-enabled system and guide the TA1 and TA2 teams to deliver corresponding design-time and operation-time evidence;

2. Develop the necessary tools and algorithms for formal representation and dynamic evaluation of assurance cases; and

3. Integrate TA1 and TA2 tools into the dynamic assurance evaluation framework. Strong proposals will also develop techniques for modularizing and automatically generating assurance cases from system design descriptions and requirements.

Research challenges in this area include, but are not limited to, development in the following areas:

Formal semantics of assurance cases that enable assessment in terms of validity and confidence;

HR001117S0045 ASSURED AUTONOMY 15

Scalable algorithms for dynamic evaluation of assurance cases consistent with the formal semantics;

Capabilities for modularizing and automatically generating assurance cases from system design descriptions; and

Application of developed capabilities to produce dynamic assurance cases for LE-CPSs.

Proposals for TA3 should address one or more of these challenges, clearly describing the innovations in the technical approach. Proposals can address other challenges not listed above, provided there is a strong justification for the centrality of the challenge to the achievement of TA3 goals. Proposals that strive to provide a comprehensive solution to these areas are encouraged, though not required. Proposals must describe at least one example of a challenge problem (a proxy for challenge problems that will be proposed by TA4) for which the proposed techniques can be applied effectively, clearly describing the metrics and capability milestones reached at the end of each Phase. The envisioned outcome of the TA3 effort is a dynamic assurance case (and the tools to build and evaluate the assurance case) that needs to be integrated into the TA4 platforms. Proposals need to explain interfaces provided for integration.

Performers in this area are expected to be part of aTA1-3 team. DARPA encourages, but doesn’t require, integrated TA1-3 proposals. Proposers submitting TA3 only proposal should anticipate joining a TA1-3 team post program kick-off, and describe their interface and plans for working with other TAs. Performers in this area will need to work closely with TA4 performers to learn about the target platforms, formulating a dynamic assurance case, and guiding the TA1 and TA2 performers. The TA3 performers will lead the effort to integrate the evidence developed by researchers in TA1 into a coherent assurance case. The TA3 performers will work with the TA4 performers to deploy the dynamic assurance case in the platform. DARPA anticipates two TA4 platforms, and the integrated TA1-3 team will need to work on challenge problems posed by both platform. To maximize flexibility for the Government, the proposers should plan to include support for one TA4 platform in the base effort, and include support for the second platform as an option.

TA4: Experimentation and Integration Platform

Each TA4 performer will produce an assured version of a single learning-enabled autonomous platform using design and analysis capabilities developed by TA1, TA2, and TA3. These platforms will serve as testbeds for experimentation, evaluation, and demonstration throughout the program. To facilitate this goal, TA4 will perform the following roles:

1. Educate other performers about use cases, and related technical challenges in producing assured versions of the platforms;

2. Develop unrestricted and unclassified challenge problems that abstract the key difficulties for use by other performers;

3. Provide design models, simulators and/or distributable prototypes and related APIs for performers of the other technical areas; and

4. Apply the research results from the other TAs to the development of the assured platform.

HR001117S0045 ASSURED AUTONOMY 16

The platforms, must be demonstrably cyber physical (such as vehicles), and should have the following characteristics:

1. DoD relevant;

2. Includes LECs or readily extensible to include LECs;

3. Accessible to performers. TA4 proposals should provide substantial details about existing, initial versions of the experimentation platforms that can be made available for other performers at the kick-off meeting to enable rapid convergence among performers.

Building on the initial capabilities, TA4 performers will also evolve the experimentation platform, extend with LECs, develop challenge problems and integrated evaluation scenarios, provide a framework and access to their platforms for allowing technology developers to integrate LECs, provide models and access to development toolchains, and experimentally validate assurance developed by TA1-3 performers.

As an example of a platform, consider an autonomous surface ship capable of navigation in open-seas. The platform includes autonomy components that allows it to detect other vessels, and navigate in compliance with COLREGS6. The platform uses learning algorithms to identify vessels and plan navigation paths consistent with COLREGS. The platform could also use learning (such as reinforcement learning) to evolve and refine its navigation behavior in harbor with dense traffic, or automatically trail other vessels when surveilling. TA4 performers are expected to develop scenarios of increasing complexity, and expanding role of learning in executing the scenarios.

Each TA4 performer will be expected to work with multiple integrated TA1-3 teams. The goal of this team will be the realization of a working end-to-end assured learning-enabled autonomous system in advance of each demonstration meeting. In this capacity, each TA4 performer is responsible for:

1. Providing the integrated design team with two platform-specific challenge problems six months after the program kick-off, as well as three platform-specific challenge problems three months after the start of Phase 2 and Phase 3;

2. Coordinating and finalizing definitions of APIs, interfaces, representations, internal domain-specific languages, and other technologies necessary to enable interaction among performers in the team;

3. Receiving and reviewing TA1, TA2, and TA3 deliverables for the challenge problems;

4. Integrating a working version of the assured platform one month before the demonstration meeting; and

5. Demonstrating the system applied to the platform-specific challenge problems at the demonstration meeting.

To be consistent with program goals, the identified platforms shared with the technology development teams (including sensors, computing hardware, and software) must be unclassified.

6 COLREGS – International Regulations for Preventing Collisions at Sea, http://www.jag.navy.mil/distrib/instructions/COLREG-1972.pdf.

HR001117S0045 ASSURED AUTONOMY 17

DARPA anticipates two TA4 platforms to evaluate and demonstrate the assurance technology, and multiple TA1-3 teams to develop the assurance technology. As such, the TA4 performer should plan to support multiple TA1-3 teams. To maximize flexibility for the Government, the TA4 proposal should plan to support two (2) integrated TA1-3 teams in the base effort, and offer support for additional TA1-3 teams as options.

E. Schedule

The schedule listed in Figure 3 contains notional estimates. Proposers should submit a detailed schedule of logically-sequenced tasks and subtasks that in sum constitute a constructive plan for achieving the proposed technical objectives while appropriately managing risk. These schedules will be synchronized across performers, as required, and monitored and reviewed as necessary throughout the Assured Autonomy program’s period of performance. For budgeting purposes, please use April 2, 2018, as an estimated start date for all technical areas.

TA1: Design for

Assurance

TA2:

Monitoring & Control

TA3: Dynamic

TA4: Integration & Experimentation

Platform

Meetings

Month A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M

Year

Phase I (18 months) Phase II (15 months) Phase III (15 months)

2018 2019 2020 2021 2022

Kick-off PI Mtg PI Mtg Phase I Demo PI Mtg

Phase II Demo PI Mtg

Capstone Demo

Research and Protoype Tools Scalability and Abstraction Techniques Tool Maturation and Integration

Exp. Support Exp. Support Demo support

Baseline Algorithm Development Algorithm Development & Integration Algorithm Maturation & Integration

Exp. Support Exp. Support Demo Support

Research and Protoype Tools Scalability and Abstraction Techniques Tool Maturation and Integration

Exp. Support Exp. Support Demo Support

CP Development CP Development CP DevelopmentIntegration & Execution Integration & Execution Integration & Execution

Figure 3: Program Schedule

The Government will specify the locations for Principal Investigator (PI) meetings, demonstration meetings, and other events. There will be two PI meetings in Phase 1, held approximately 6 months and 12 months after the kick-off meeting. There will be one PI meeting in both Phase 2 and Phase 3, held roughly 7.5 months from the beginning of each phase. PI meeting locations are likely to be split between performer locations. The goals of the PI meetings will be to primarily present new research findings and accomplishments, review plans for the next period, discuss implementation milestones, and resolve any programmatic, budgeting, or logistics issues. Demonstration meetings will be held at the end of each phase.

The goal of the end-of-phase demonstration meeting is to provide demonstrable evidence and evaluation of the assurance technologies to the Government and other stakeholders.

For travel costing, assume 8 trips during the entire 3 phases (2018-2022) per the program schedule shown above, alternating between Washington, DC and San Diego, CA with each trip requiring 3 days and 2 nights. .

HR001117S0045 ASSURED AUTONOMY 18

In addition to site visits, monthly teleconference meetings will be held with each PI to enhance communications with the Government team.

F. Milestones

Six months after program kick-off, each TA4 performer will finalize platform-specific challenge problems (two in Phase 1, three in Phase 2, and three in Phase 3) that they will be expected to develop their systems against and present during the demonstration meetings, which will be held at the end of each phase. One month prior to the demonstration meeting, each TA4 performer working with the TA1-3 integrated teams will produce a working version of the integrated platform. During the demonstration meeting, the TA4 performers will demonstrate their system on challenge problems developed for that phase and initiate discussions on proposed platform-specific challenge problems for the following phase.

G. Deliverables

TA1, TA2, and TA3 performers will deliver appropriate code and documentation to the TA4 performer that they are working with, at regular intervals, and on a schedule consistent with the delivery of a working platform one month before each demonstration meeting. It is expected that the TA4 performers will be in close collaboration with the TA1, TA2, and TA3 performers at least one (1) month before each demonstration meeting to ensure program deliverables are met.

In addition, all performers will be required to provide the following deliverables:

Technical papers covering work funded by Assured Autonomy;

Source code, Algorithm and Interface Description Document, user guides, other necessary data, and documentation, assumptions and limitations for all software developed under this program;

Slide Presentations. Annotated slide presentations shall be submitted within one month after the program kick-off meeting and after each program event (program reviews, PI meetings, and technical interchange meetings);

Quarterly Progress Reports. A quarterly progress report describing technical progress made, resources expended, major risks, planned activities, trip summaries, changes to key personnel, and any potential issues or problem areas that require the attention of the Government team shall be provided within 10 days after the end of each quarter;

Monthly Progress Reports. A monthly progress report in the form of a PowerPoint document describing technical progress, planned activities for next month, any technical, financial, and programmatic issue shall be provided and presented in a teleconference with DARPA;

Monthly financial status reports;

A final phase report after each program phase. The final report shall concisely summarize the effort conducted; and Final Technical Report.

H. Intellectual Property

As mentioned earlier, Assured Autonomy will emphasize creating and leveraging open source technology. Intellectual property rights asserted by proposers are strongly encouraged to be aligned with open source regimes. See Section VI.B.1 for more details.

HR001117S0045 ASSURED AUTONOMY 19

II. Award Information

A. Awards

Multiple awards are anticipated. The level of funding for individual awards made under this solicitation has not been predetermined and will depend on the quality of the proposals received and the availability of funds. Awards will be made to proposers whose proposals are determined to be the most advantageous to the Government, all factors considered, including the potential contributions of the proposed work, overall funding strategy, and availability of funding. See Section V for further information.

The Government reserves the right to:

select for negotiation all, some, one, or none of the proposals received in response to this solicitation;

make awards without discussions with proposers;

conduct discussions with proposers if it is later determined to be necessary;

segregate portions of resulting awards into pre-priced options;

accept proposals in their entirety or to select only portions of proposals for award;

fund proposals in increments and/or with options for continued work at the end of one or more phases;

request additional documentation once the award instrument has been determined (e.g., representations and certifications); and remove proposers from award consideration should the parties fail to reach agreement on award terms within a reasonable time or the proposer fails to provide requested additional information in a timely manner.

Proposals selected for award negotiation may result in a procurement contract or Other Transaction (OT) depending upon the nature of the work proposed, the required degree of interaction between parties, and other factors.

Proposers looking for innovative, commercial-like contractual arrangements are encouraged to consider requesting Other Transactions. To understand the flexibility and options associated with Other Transactions, consult www.darpa.mil/work-with-us/contract-management#OtherTransactions.

In all cases, the Government contracting officer shall have sole discretion to select award instrument type, regardless of instrument type proposed, and to negotiate all instrument terms and conditions with selectees. DARPA will apply publication or other restrictions, as necessary, if it determines that the research resulting from the proposed effort will present a high likelihood of disclosing performance characteristics of military systems or manufacturing technologies that are unique and critical to defense. Any award resulting from such a determination will include a requirement for DARPA permission before publishing any information or results on the program. For more information on publication restrictions, see the section below on Fundamental Research.

http://www.darpa.mil/work-with-us/contract-management#OtherTransactions http://www.darpa.mil/work-with-us/contract-management#OtherTransactions

HR001117S0045 ASSURED AUTONOMY 20

B. Fundamental Research

It is DoD policy that the publication of products of fundamental research will remain unrestricted to the maximum extent possible. National Security Decision Directive (NSDD) 189 defines fundamental research as follows:

‘Fundamental research’ means basic and applied research in science and engineering, the results of which ordinarily are published and shared broadly within the scientific community, as distinguished from proprietary research and from industrial development, design, production, and product utilization, the results of which ordinarily are restricted for proprietary or national security reasons.

As of the date of publication of this BAA, the Government expects that program goals as described herein may be met by proposers intending to perform fundamental research and proposers not intending to perform fundamental research or the proposed research may present a high likelihood of disclosing performance characteristics of military systems or manufacturing technologies that are unique and critical to defense. Based on the nature of the performer and the nature of the work, the Government anticipates that some awards will include restrictions on the resultant research that will require the awardee to seek DARPA permission before publishing any information or results relative to the program.

Proposers should indicate in their proposal whether they believe the scope of the research included in their proposal is fundamental or not. While proposers should clearly explain the intended results of their research, the Government shall have sole discretion to select award instrument type and to negotiate all instrument terms and conditions with selectees. Appropriate clauses will be included in resultant awards for non-fundamental research to prescribe publication requirements and other restrictions, as appropriate. This clause can be found at www.darpa.mil/work-with-us/additional-baa.

For certain research projects, it may be possible that although the research being performed by the awardee is restricted research, a subawardee may be conducting fundamental research. In those cases, it is the awardee’s responsibility to explain in their proposal why its subawardee’s effort is fundamental research

C. Disclosure of Information and Compliance with Safeguarding Covered Defense Information Controls

The following provisions and clause apply to all solicitations and contracts; however, the definition of “controlled technical information” clearly exempts work considered fundamental research and therefore, even though included in the contract, will not apply if the work is fundamental research.

DFARS 252.204-7000, “Disclosure of Information” DFARS 252.204-7008, “Compliance with Safeguarding Covered Defense Information Controls” DFARS 252.204-7012, “Safeguarding Covered Defense Information and Cyber Incident Reporting”

The full text of the above solicitation provision and contract clauses can be found at http://www.darpa.mil/work-with-us/additional-baa#NPRPAC.

http://www.darpa.mil/work-with-us/additional-baa http://www.darpa.mil/work-with-us/additional-baa#NPRPAC

HR001117S0045 ASSURED AUTONOMY 21

Compliance with the above requirements includes the mandate for proposers to…