HR001119S0083.pdf

Broad Agency Announcement Symbiotic Design for Cyber Physical Systems

HR001119S0083

August 13, 2019

Defense Advanced Research Projects Agency Information Innovation Office 675 North Randolph Street Arlington, VA 22203-2114

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 2

Table of Contents Part I: Overview Information……………………………………………………………………...………4

Part II: Full Text of Announcement………………………………………………………………………..5

I. Funding Opportunity Description

A. Introduction and Background

B. Program Description

C. Technical Areas

D. Evaluation Phases and Schedule

E. Metrics

F. Deliverables

G. Intellectual Property

II. Award Information

A. Awards

B. Fundamental Research

C. Disclosure of Information and Compliance with Safeguarding Covered Defense Information Controls

III. Eligibility Information

A. Eligible Applicants

B. Organizational Conflicts of Interest

C. Cost Sharing/Matching

D. Other Eligibility Requirements

IV. Application and Submission Information

A. Address to Request Application Package

B. Content and Form of Application Submission

C. Submission Dates and Times

D. Funding Restrictions

E. Other Submission Requirements

V. Application Review Information

A. Evaluation Criteria

B. Review and Selection Process

VI. Award Administration Information

A. Selection Notices

B. Administrative and National Policy Requirements

C. Reporting

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 3

VII. Agency Contacts

VIII. Other Information

A. Frequently Asked Questions (FAQs)

B. Proposers Day

C. Submission Checklist

D. Associate Contractor Agreement (ACA)

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 4

PART I: OVERVIEW INFORMATION

• Federal Agency Name: Defense Advanced Research Projects Agency (DARPA), Information Innovation Office (I2O)

• Funding Opportunity Title: Symbiotic Design for Cyber Physical Systems

• Announcement Type: Initial Announcement

• Funding Opportunity Number: HR001119S0083

• Catalog of Federal Domestic Assistance Numbers (CFDA): Not Applicable

• Dates o Posting Date: August 13, 2019 o Proposers Day: August 12, 2019 o Abstract Due Date: August 29, 2019, 12:00 noon (ET) o Proposal Due Date: October 14, 2019, 12:00 noon (ET) o BAA Closing Date: October 14, 2019, 12:00 noon (ET)

• Anticipated Individual Awards: DARPA anticipates multiple awards in technical areas

(TAs) 1 and 2 and no more than two awards in TA3

• Types of Instruments that May be Awarded: Procurement contracts or Other Transactions (OT)

• Agency Contacts o Technical POC: Dr. Sandeep Neema, Program Manager, DARPA/I2O o BAA Email: SymbioticDesign@darpa.mil o BAA Mailing Address:

DARPA/I2O

ATTN: HR001119S0083

675 North Randolph Street Arlington, VA 22203-2114 o I2O Solicitation Website: http://www.darpa.mil/work-with-us/opportunities mailto:SymbioticDesign@darpa.mil http://www.darpa.mil/work-with-us/opportunities

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 5

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

DARPA is soliciting innovative research proposals in the area of AI-based approaches for design of military-relevant cyber-physical systems (CPS). Proposed research should investigate innovative approaches that enable revolutionary advances in science, devices, or systems.

Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice.

This Broad Agency Announcement (BAA) is being issued, and any resultant selection will be made, using procedures under Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016.

Any negotiations and/or awards will use procedures under FAR 15.4. Proposals received as a result of this BAA shall be evaluated in accordance with evaluation criteria specified herein through a scientific review process.

DARPA BAAs are posted on the Federal Business Opportunities (FBO) website (https://www.fbo.gov/)

The following information is for those wishing to respond to this BAA.

A. Introduction and Background The goal of the Symbiotic Design for CPS program is to develop AI-based approaches to enable correct-by-construction design of military-relevant CPS, in order to reduce the time from their inception to deployment from years to months, and enhance innovation in design.

CPS are instrumental to current and future Department of Defense (DoD) mission needs – unmanned vehicles, weapon systems, and mission platforms are all examples of military-relevant CPS. These systems and platforms integrate cyber and physical subsystems, and the enormous complexity of the resulting CPS has made their engineering design a daunting challenge. An immediate consequence of this complexity is development cycles with prolonged timelines that challenge DoD’s ability to counter emerging threats.

CPS design is a complex endeavor that involves many domains1 and hundreds of domain-specific tools orchestrated by large teams of engineers with extensive domain knowledge and subject matter expertise. Current engineering design processes start with a requirement-driven decomposition into discipline-specific design flows that at their core are concurrently running sequential decision-making processes. They involve generating candidate architectures, evaluating, selecting, and refining options and integrating the design until requirements are satisfied.

Three intrinsic challenges contribute to the long design cycles, costly redesigns, and hamper innovation today:

(a) Predictability – The soundness of design decisions relies on accurate predictions of performance prior to the implementation of software and physical components. However, 1 Examples for domains include cyber (e.g. software, control, computing and communication), physical (e.g., structural, mechanical, thermal, etc.), and manufacturing https://www.fbo.gov/

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 6

accurate predictions require high-fidelity models that are cost- and time-prohibitive to produce. Cost-effective modeling processes produce results with substantial uncertainty.

(b) Convergence – Design teams are federated according to discipline boundaries. However, separation of concerns within a complex system neglects the interdependence of design decisions, rendering rapid convergence to a viable integrated solution practically impossible.

(c) Exploration – Limited by time and resources, engineers have to make tradeoffs that constrain the exploration of the design space to the familiar and known-feasible. This leaves vast areas of design space unexplored, which may contain unconventional but highly performant solutions.

The aspirational goal in CPS design is to be “correct-by-construction,” which refers to the ability to predict the system’s properties with high confidence prior to construction. The correct-by-construction approach to design is typically model-based. However, achieving full correct-by-construction remains elusive due to practical and fundamental limitations in modeling: the cost of creating models, the heterogeneity of models, and the knowledge gaps about physical and computational processes involved. The state-of-practice consequently is “construct-by-correction,” in which integration events surface errors that cause multiple redesign iterations.

Decomposition of the design into relatively isolated design concerns is unavoidable, not only to manage complexity of design flows, but also because existing design disciplines are the primary way of accumulating reusable design knowledge through domain-specific engineering tools and libraries. Implicit in this decomposition is a presumed separation of concerns that does not reflect reality, because the domains in CPS are intrinsically coupled, and design decisions made in one domain unavoidably and intractably affect other domains. These unmodeled and unanticipated interactions often manifest late in design cycle or in the initial prototypes, resulting in expensive design changes.

The lack of fully automated design flows and knowledge of cross-domain interactions forces designers to limit the size of the design space. Committing early to a limited set of design alternatives reduces the chance of discovering breakthrough designs. Recent advances in experimental CPS design flows incorporate automated composition of analysis models, but there are no known methods for adaptive and incremental search strategies in sufficiently large, heterogeneous multi-domain design spaces.

The vision of the program is to vastly expand coverage and accelerate exploration of CPS design spaces with the symbiosis of two very different kind of agents: humans with their uncanny ability to create intuitive associations across design domains, and machines with their ability to recognize statistical patterns from data and navigate vast search spaces for optimal solutions. The program will realize this vision by transforming the human-focused model-based design flows used today into a symbiotic process of collaborative discovery by humans and continuously-learning AI-based co-designers.

The technologies developed in the program will have direct applications to DoD in two distinct contexts – rapid mission-specific system design, as well as traditional large-scale mission systems design. Rapid Equipping Forces, and Special Forces, need new capabilities on mission timescales. These design cycles are atypical of traditional DoD contracts and acquisition, and place a premium on agile, rapid, and innovative but verified designs. Introduction of AI techniques in CPS design process will be a game changer, and may result in a new generation of

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 7

unexpected, counterintuitive design solutions – similarly to unexpected, counterintuitive moves of AI-based game engines. We expect order of magnitude improvement in design productivity, but equally important, the appearance of surprises, in the discovery of unconventional but highly performant designs.

B. Program Description The overarching goal of the program is to develop AI-based approaches to complement and augment existing model-based design technologies, and enable humans and computers to collaborate on correct-by-construction design of CPS. Managing multi-domain interactions is a critical challenge in CPS, and the focus of the program is on developing technologies that enable concurrent exploration and co-design across multiple design domains.

The program envisions that the AI-based approaches will be integrated together, manifesting as an “AI co-designer” that enables the following (notional) symbiotic design flow:

• Human designers communicate their intent via domain-specific design artifacts, such as seed designs, design fragments, or abstract designs, in addition to traditional requirement specifications such as performance, functional, and other –ility objectives (e.g. reliability, maintainability, serviceability, etc.);

• The AI co-designer, learning from past successful designs and design corpora, and based on explicit and inferred specification of the design problem, proposes a set of refinement alternatives to construct a multi-domain design-space for the problem under consideration. Human designers may accept, reject, or expand the set of proposed refinement alternatives;

• The AI co-designer, with assistance from human designers, composes points within the design space, elaborates and augments the design points with inferred cross-domain interactions, and automates the accelerated evaluation of design points using domain-specific analysis and simulation tools; and

• The AI co-designer analyzes the evaluation results and explores the design space, with the human designer providing strategies for refinement and navigation of the design space.

Figure 1: Notional UUV Design Space

N : number of subsystems C : number of components per subsystem M : number of options per component

Combinatorically Large Design

Space

UUV

Design Problem

UUV Architecture with representative subsystems

𝐷𝑒𝑠𝑖𝑔𝑛 𝑆𝑝𝑎𝑐𝑒 = 𝑀𝑐 𝑁

(Simplified example with same C and M for each subsystem)

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 8

To exemplify this flow, consider a vignette conducting the design of a notional UUV depicted in Figure 1 (system) and Figure 2 (design flow).

Figure 1 is an architectural view of a notional UUV, along with a few representative subsystems and components that span multiple design domains – for example, hull form, navigation sensors, control algorithms, mission payloads, and battery. The design space for such a system, even when considering only alternative implementations of subsystems and components is combinatorically large. Further, the selection decisions across these subsystems are closely coupled. The choice of hull form determines the dynamics of the vehicle, and consequently the control algorithm. The choice of desired mission payloads and their placement is coupled with the hull form that constrains the internal layout. The desired mission profile, the sensors and payloads, and propulsion power requirements determine the battery type and size, which in turn couples with placement and thermal management. These interactions and couplings need to be accounted for in the modeling and the resulting multi-domain design spaces need to be jointly explored.

Figure 2 illustrates the envisioned design flow, described below:

(1) The design flow begins with human designers providing design specifications (performance objectives, dimension targets, mission payloads, etc.) for the UUV, as well as initial seed design(s). The seed design includes an architectural description with major subsystems (such as hull, navigation sensors, control algorithms, etc.,) and components as well as relevant domain models.

(2) The AI co-designer extracts features from the subsystems and components in the seed design, and finds functionally equivalent components and subsystems in a design corpus of past designs. Human engineers refine the output of the AI co-designer, adding options or removing non-viable options based on their design experience and expertise. The outcome is a design space that includes options for subsystems and components.

(3) The AI co-designer and human engineers jointly identify regions and points within the design space for composition and evaluation. A design point is a selection of one option

Figure 2: Overview of Symbiotic Design Flow for a notional UUV design sp ee d( s) range (r)

Exploration 𝑠, 𝑟 = 𝑓(ℎ,𝑛, 𝑐,𝑚, 𝑏)

Seed Design

Design Space

Evaluation Results

Models 𝑠 = 𝑠𝑝𝑒𝑒𝑑 ;𝑟 = 𝑟𝑎𝑛𝑔𝑒; ℎ = ℎ𝑢𝑙𝑙 𝑓𝑜𝑟𝑚;

𝑛 = 𝑛𝑎𝑣𝑖𝑔𝑎𝑡𝑖𝑜𝑛 𝑠𝑒𝑛𝑠𝑜𝑟𝑠;𝑐 = 𝑐𝑜𝑛𝑡𝑟𝑜𝑙 𝑎𝑙𝑔𝑜𝑟𝑖𝑡ℎ𝑚;

𝑚 = 𝑚𝑖𝑠𝑠𝑖𝑜𝑛 𝑝𝑎𝑦𝑙𝑜𝑎𝑑𝑠; 𝑏 = 𝑏𝑎𝑡𝑡𝑒𝑟𝑦

Design Space ~ 1030

Humans provide

Computer mines design corpora Humans curate design options

Computer composes domain models

Humans provide rules and examples

Computer simulates and analyzes models

Computer learns f , searches for “best” design

Humans guide search

Hull Form Battery Control

Algorithm Navigation Sensors

Mission Payloads

UUV Variant

(1)

(2)

(3)

(4)

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 9

for each subsystem. Composed models are evaluated in domain-specific engineering tools, using provided evaluation scenarios. The evaluation takes places in multiple tools – for example, computer-aided design (CAD) tools may be used to calculate inertial matrices, which may be used in dynamics models for evaluation of control performance, while power and thermal models may be used to evaluate battery performance. The aggregated evaluation results are high-level performance objectives (e.g. speed and range for the notional UUV).

(4) The AI co-designer learns the mapping from design space to performance space (function f in the Figure 2), and identifies the next design points to sample to find a “best” design.

The learning of this mapping takes place during exploration, however, the AI co-designer can leverage background knowledge learned offline to accelerate the exploration. Human designers modulate the selection of design points, steering the exploration towards feasible and optimal regions. The outcome of the exploration is the digital representation of a design (digital twin) that is close to optimal with respect to the desired specifications.

The program will address several hard technical challenges to develop the core capabilities of the AI co-designer: design space construction, design composition, design space exploration, and symbiotic exchange.

• Design Space Construction – The program seeks technologies to generate design options for a given design problem. The challenge for Symbiotic Design is to develop approaches for mining heterogeneous engineering domain artifacts to discover a viable space of potential solutions.

• Design Composition – The program seeks technologies to compose models for a set of chosen design options to enable evaluation in domain-specific engineering tools.

Previous approaches to model composition relied on a prescribed set of rules for automatic derivation of engineering domain artifacts from architectural descriptions. The challenge is to enable the learning of synthesis rules from data and examples to allow design composition at scale necessary for Symbiotic Design.

• Design Space Exploration - The program seeks technologies to explore high-dimensional multi-domain design spaces that are typical of CPS. The challenge is to develop scalable, adaptive and incremental search strategies that can efficiently explore heterogeneous multi-domain design spaces, and are amenable to guidance and steering by human designers.

• Symbiotic Exchange – The program seeks technologies to enable visualization and manipulation of high-dimensional design spaces, as well as natural, context-aware, and anticipatory interfaces to facilitate human machine collaboration.

In order to concretize and ground the research objectives, the program will prioritize challenge problems in CPS that are relevant to DoD. However, it is anticipated that the tools, toolchains, and algorithms created will be relevant to other CPS. The resulting technology from the program will be in the form of a set of publicly available tools integrated into a symbiotic design environment that will be made widely available for use in commercial and defense sectors.

C. Technical Areas The program has three technical areas (TA).

• TA1: AI Co-Designer

• TA2: Symbiotic Exchange

• TA3: Program Evaluation

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 10

As shown in Figure 3, the AI co-designer integrates the three core technologies (Design Space Construction, Composition, and Exploration) in an iterative loop, which starting from baseline seed design(s) extends iteratively and incrementally towards a design space. Using composition and exploration technologies, the co-designer will navigate through this design space, to produce functionally correct, innovative solutions.

Humans will collaborate with the AI co-designer through the symbiotic exchange, by jointly participating in the execution of this loop, shaping the design space, analyzing the points in the design space, and steering the AI co-designer away from unproductive regions of the design space. Visualizations of the design space as it evolves will support this human interaction.

The AI co-designer and Symbiotic Exchange are general technologies that are instantiated and adapted for problem instances with data, which may include specifications for the new design as well as prior design knowledge. Seed designs are the collection of proven or relevant prior designs that are most related to the new design. Prior design knowledge may be in the form of model libraries, component libraries, design rules, and/or constraints. The program may leverage open source content and/or proposers may propose the use of their own existing corpora. Finally, design problem specification of the new system will include not just the requirements but also the associated tests and scenarios through which to evaluate the requirements.

There are multiple points of essential collaboration among TAs, and the Government expects all performers to collaborate effectively. Proposers should read the descriptions of all TAs and the Program Assessments/Schedule section to ensure a full understanding of the program context, structure, and anticipated relationships required among performers. To facilitate the open exchange of information, all program performers will have Associate Contractor Agreement (ACA) language included in their award. See Section VIII for further information. Each proposal may only address a single TA, but proposers may submit multiple proposals. However, proposers selected for award in TA3 as prime will not be selected for award in any other TA.

Figure 3: Symbiotic Design Program Architecture and Technical Areas 1-3

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 11

TA1: AI Co-Designer This TA has three sub TAs that together will develop closely coupled technologies to implement the AI co-designer: (1) TA1.1 – Design Space Construction, (2) TA1.2 – Design Composition,

(3) TA1.3 – Design Space Exploration. The technologies developed in the sub TAs, while spanning distinct research disciplines, will need to interoperate seamlessly to enable a tightly integrated design space construction, composition, and exploration loop.

TA1.1: Design Space Construction The goal of TA1.1 is to develop technologies to construct design spaces for a given (partial or complete) design problem and seed designs. The seed designs may be in the form of architecture models of a system (such as SysML/CyPhyML) describing the subsystems and components of the system as well as interaction between the components. The seed designs may also include domain-specific models of a system capturing the design of different domains or subsystems (such as a Simulink model of the thermal management subsystem). The design corpora, similarly, may include heterogeneous set of artifacts – architecture models, dynamics models, CAD models, source code, design specifications, component meta-data, operation logs, problem reports. TA3 and TA1 performers will collaboratively determine the formats of the design artifacts to be pursued in the program, and the proposers in TA1.1 are encouraged to propose approaches that are flexible to accommodate diverse modeling formalisms and formats.

Research challenges in TA1.1 include, but are not limited to, development in the following areas:

• Query generation from design problem and seed designs – the proposed technologies should enable extraction of latent specifications from seed designs to generate queries for the mining engines.

• Mining engine for heterogeneous design artifacts – the proposed technologies should enable mining of heterogeneous design artifacts. The proposed technologies should also enable learning design rules and constraints from design artifacts.

• Incremental construction of design space – the proposed technologies should enable incremental construction of the design spaces for identified subsystems in the design.

TA1.2: Design Composition The goal of TA1.2 is to develop technologies to automatically compose and evaluate a design point. Composition refers to assembling a complete model for the system for a specific analysis in a specific engineering tool. It is expected that the design spaces TA1.1 performers produce will identify options for different components and subsystems. However, evaluation of these options may require a complete system model that instantiates the options, and provides the requisite context. The architecture and domain-specific models of the seed design may serve as a partial/incomplete context that can be utilized to construct a complete system model with the options to be evaluated. The completion process should also reason about potential cross-domain interactions (such as thermal or acoustic effects), and introduce additional elements into the models to account for these interactions. The proposed technologies should enable inference of cross-domain interactions from observations of behavior of computational and physical artifacts.

Evaluations in engineering tools, either using simulations or higher-fidelity analysis (partial differential equations (PDE), finite element analysis [FEA]) are computationally expensive, and proposers are encouraged to propose algorithmic approaches that can accelerate such evaluations.

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 12

Research challenges in TA1.2 include, but are not limited to, development in the following areas:

• Automated model completion in heterogeneous domains – the proposed technologies should enable automated completion of partial models in domain-specific modeling tools (such as CAD, Simulink, Modelica, etc.). The degree of incompleteness may range from missing connections, to missing interface components and functional components. The proposed technologies should be able to infer and complete the models based on underlying physics, semantics of the modeling formalism, context, as well as prior examples.

• Automated cross domain reasoning and model learning – the proposed technologies should enable analysis of multiple domain-specific models and infer cross-domain interactions – for example, a higher fidelity analysis, such as thermal FEA, may reveal coupling between diverse components, which can then be abstracted and represented accurately in a lower-fidelity analysis. Some cross-domain interactions may be represented with simple analytic models, while others may require data-driven or machine-learned surrogates.

• Accelerated design analysis and simulation – the proposed technologies should enable algorithmically accelerated evaluation of designs.

TA1.3: Design Space Exploration The goal of TA1.3 is to develop technologies to explore high-dimensional, multi-domain combinatorial design spaces. Conceptually, design space exploration is a constrained optimization problem that optimizes a set of objective functions, defined over a set of design parameters, while satisfying a set of constraints. The challenge in CPS design space exploration, however, emerges from multiple factors.

1) the dimensionality of the space, a combinatorial product of the number of domains, number of components and subsystems, number of options per component, and number of parameters per component;

2) the computational complexity of evaluation of the objective function, which in most cases is a simulation (DE, ODE, PDE, etc.) or a complex analysis (FEA); and

3) the construction of objective function requires composition of a model that is specific to the selected set of design options, and trades between the fidelity and abstractions in the models employed.

Research challenges in TA1.3 include, but are not limited to, development in the following areas:

• methods that tackle high-dimensionality of the space;

• methods that support heterogeneous domain models;

• objective functions that are defined over multiple abstraction layers.

Proposals for TA1 should address one or more of these sub TAs, clearly describing the innovations in the technical approach. Proposals can address other challenges not listed above, provided there is a strong justification for the centrality of the challenge to the achievement of TA1 goals. Proposals for a comprehensive solution to all three sub-TAs TA1.1, TA1.2, and TA1.3, are strongly encouraged, though not required. Proposers in this TA should clearly identify and organize statement of work tasks by sub TAs. Proposals that do not address all the sub TAs, must clearly identify and describe interfaces for other sub TAs, and should describe a plan to participate in an integrated TA1 team composed of different performers.

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 13

Proposals must describe at least one example of a challenge problem (as a proxy for challenge problems that will be proposed by TA3), and explain how the proposed techniques can be applied effectively on a multi-domain design problem. Proposers must describe the relevant metrics and the capability milestones reached at the end of each phase.

TA1 performers are expected to work closely with TA2 and TA3 performers. Collaboration with TA3 performers will be necessary to learn about the design challenge problem, any provided design artifacts or knowledge bases, and to consult on the application of their techniques to the target problem. The output from TA1 will be a digital representation of the design (digital twin), and collaboration with TA3 will be necessary to define the languages and formats for the digital representation. Collaboration with TA2 performers will be necessary to develop the appropriate user interfaces to enable symbiosis. TA1 performers must demonstrate their tools on appropriate portions of the challenge problems, and provide the ability to derive a complete design satisfying the performance objectives.

DARPA anticipates that the TA3 performers will provide multiple challenge problems that will scale up in complexity across the program phases, and expect that the TA1 technologies to address all the challenge problems.

TA2: Symbiotic Exchange The goal of TA2 is to build the interface for partnership between human designers and AI co-designer. This partnership spans all stages of automation: construction, composition, and exploration. Humans supervise and refine design space and support model composition (in TA1.1 and TA1.2). Additionally, humans shape the AI’s reward (or optimization) function during exploration, to avoid exploring design space not considered viable by human designers based on their experience. The human guidance to constrain the design space exploration is essential, however it must not be over-constraining to preclude discovery of counter-intuitive, but highly performant designs. Some of the specific symbiotic exchanges include, but are not limited to the following:

In TA1.1:

• Human to Machine: From visualization of design space, add constraints or rules for options mined by the AI-co-designer. Input maybe in the form of design artifacts or sketches.

• Machine to Human: Provide visualizations of mined designs space; potentially rank the likely salient or critical aspects of the design factors. Anticipate salient features from past interactions.

In TA1.2:

• Human to Machine: Provides examples (in the form of artifacts or sketches) for example-driven inductive learning of model composition for the problem at hand.

• Machine to Human: a) Provide visualizations of a composed model, and the rules employed for composition; b) performance visualization from high-dimensional data;

c) anticipate salient features from past interactions.

In TA1.3:

• Human to Machine: Shape the reward function for AI exploration of design space, based on reported algorithm performance in a specific time interval. For example: a)

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 14

no solution found; subject matter expert (SME) sets a different initialization; b) algorithm finds the same constraint being violated in many runs, such as weight or a power limit; SME guides the start from a different region of design space; c) SME suggests a non-symmetric design feature; d) SME suggests evaluation at a higher resolution.

• Machine to Human: Provide visualizations of algorithm search performance for a given interval. Anticipate salient features from past interactions.

Research challenges in this TA include, but are not limited to, development in the following areas:

• visualization and understanding of high-dimensional design spaces;

• methods for shaping and guiding exploration;

• methods for mitigating interaction complexity of engineering design tools.

Proposals for TA2 should address one or more of these challenges, clearly describing the innovations in the technical approach. Proposals can address other challenges not listed above, provided there is a strong justification for the centrality of the challenge to the achievement of TA2 goals. Proposals for a comprehensive solution to these areas are encouraged, though not required. Proposals must describe at least one example of a challenge problem (as a proxy for challenge problems that will be proposed by TA3), and explain how the proposed techniques can be applied effectively, clearly describing the metrics and capability milestones reached at the end of each phase.

TA2 performers are expected to work closely with TA1 and TA3 performers. Collaboration with TA3 performers will be necessary to learn about the design challenge problem, any provided design artifacts or knowledge bases, and to consult on the application of their techniques to the target problem. Collaboration with TA1 performers will be necessary to develop interfaces and integrate with the tools developed by the TA1 performers. TA2 performers must demonstrate their tools on appropriate portions of the challenge problems, and support assessment of the symbiosis technologies.

DARPA anticipates that the TA3 performers will provide multiple challenge problems that will scale up in complexity across the program phases, and expect that the TA2 technologies to address all the challenge problems.

TA3: Program Evaluation The TA3 performers will provide challenge problems to validate and evaluate the design technologies developed by the TA1 team, and the symbiosis technologies developed by TA2 teams. Proposed design problems should be relevant to DoD concerns and push the envelope in terms of performance objectives. The TA3 performers must provide a surrogate version of the challenge problem that has no direct military relevance to enable fundamental research.

It is expected that data (seed designs, design corpus, design problems) to be shared with TA1 and TA2 performers will be accessible with no CTI (Controlled Technical Information). The TA3 performers may also perform demonstrations of the Symbiotic Design technologies on military-relevant designs, for which the underlying data may include CTI (for specifications, design artifacts, or test environments). Such demonstrations must be managed solely by the TA3 performer.

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 15

Performers in this TA will:

• develop challenge problems that serve as surrogate/proxy of DoD relevant design problems;

• develop system specifications, requirements (performance envelope), and evaluation metrics for challenge problems;

• develop and provide performer accessible design corpora for different design domains;

• evaluate the design provided by TA1 performers, and interaction technologies developed by the TA2 performers. The design produced by TA1 will be in the form of a digital twin, a digital blueprint of the system;

• (optionally) build and test a physical prototype of the design in Phase 3.

The design corpora will be critical for the technologies being developed by the TA1 performers, and the proposers should clearly describe a plan to gather and curate a corpus of design artifacts most relevant to the challenge problems. The design corpus should include, but not be limited to, architecture models, dynamics models, CAD models, source code, design specifications, component meta-data, operation logs, and problem reports. The proposers should articulate their approach to leverage open-source artifacts where possible, and supplement with challenge problem specific design data.

Performers in this TA will assess the program technologies at two levels:

• the productivity and effectiveness of the human-machine partnership to develop the CPS using the new tools; and

• the performance and innovativeness of the CPS that is being designed using the tools developed in the program.

The productivity and effectiveness of the human-machine partnership will be evaluated using short design hackathons. The hackathon evaluation should include appropriately scoped design challenges (these may be distinct from the challenge problems, or may be a scaled-down version of the challenge problem), in which a team of engineers using traditional design approaches can be competed against a team paired with the AI-codesigner (see evaluation metrics in section E).

TA3 performers are responsible for organizing and hosting design hackathons, which are expected to be two weeks in duration, per the schedule in section D. TA3 performers are responsible for allocating engineering resources to execute the design challenges using the conventional engineering tools. TA1 and TA2 performers are responsible for supporting and participating in the design hackathons. Evaluation tasks requiring HSR (Human Subject Research) should be separately identified in the Statement-of-Work (SOW), and separately costed in the Cost Volume of the proposal.

The performance and innovativeness of the CPS that is being designed will be assessed in the context of the larger challenge problems. The TA3 proposers should describe their approach to evaluate the program technologies with respect to the metrics in section E.

D. Program Phases and Schedule The Symbiotic Design program is organized in three phases. Phase 1 will be eighteen (18) months, and will consist of initial research and tool development. Phase 2 will be fifteen (15) months, and will focus on expanding the scope of application and scaling. Phase 3 will also be fifteen (15) months, and will focus on technology maturation and capstone demonstration. Each

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 16

phase will end with demonstrations of the tools on the selected challenge problems, which will provide an opportunity to evaluate the progress made against the program objectives. (See Figure 4.) Proposers should submit a detailed schedule of logically sequenced tasks and subtasks that in sum constitute a constructive plan for achieving the proposed technical objectives while appropriately managing risk. Schedules will be synchronized across performers, as required, and monitored and reviewed throughout the Symbiotic Design program’s period of performance. For budgeting purposes, use April 2, 2020, as a start date for all TAs.

Six months after program kick-off, TA3 performers will finalize design challenge problems and provide the design data (seed designs, and design corpus) to the TA1 and TA2 performers. The TA1 performers will solve the challenge problem over the course of the phase using their technologies, and present their result during the demonstration meetings, which will be held at the end of each phase. One month prior to the demonstration meeting, the TA3 performer will work closely with the performers TA1 and TA2 to support integration and evaluation. During the demonstration meeting, the TA3 performers will demonstrate the TA1 solutions to the challenge problems developed for that phase and initiate discussions on proposed specific challenge problems for the following phase. The goal of the end-of-phase demonstration meeting is to provide demonstrable evidence and evaluation of the design technologies to the Government and other stakeholders.

Except for the first PI meeting, design hackathons will be held approximately one month prior to each PI meeting. The goal of these hackathons will be to evaluate the intermediate progress of TA1 and TA2 technologies, and perform comparative assessment of the program-developed technologies with conventional engineering tools. The hackathons should be arranged by TA3 performers at a suitable site, and attended by technical developers from TA1 and TA2 teams.

The TA3 performers should assume this responsibility when pricing the proposal, and in the event there are more than one TA3 performer, the Government will decide which one is responsible and negotiate cost changes accordingly. The results from the hackathon will be discussed at the following PI meeting.

Figure 4: Program Schedule

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 17

The Government will specify the locations for Principal Investigator (PI) meetings. There will be two PI meetings in Phase 1, held approximately 6 months and 12 months after the kick-off meeting. There will be one PI meeting in both Phase 2 and Phase 3, held roughly 7.5 months after the beginning of each phase. PI meeting locations are likely to be spread across performer locations, and the proposers should plan to host at least one PI meeting over the duration of the program. The goals of the PI meetings will be to present new research findings and accomplishments, review plans for the next period, discuss implementation milestones, and resolve any programmatic, budgeting, or logistics issues.

In addition to these program-wide events, the Government team will conduct site visits, and will hold monthly teleconference meetings with each PI to enhance communications with the Government team.

For travel planning and costing, assume 8 trips during the entire 3 phases (2020-2024) per the program schedule shown above, alternating between Washington, DC and San Diego, CA, with each trip requiring 3 days and 2 nights. Additionally, the Hackathon preceding the PI meeting will be two weeks in duration and should be attended by key technical developers of the performer teams. The TA3 proposers should plan to host the hackathon meeting, taking into consideration participation of foreign nationals. The TA1 and TA2 proposers should price their travel assuming Washington DC as a location, though the actual location will be determined by the TA3 performer, and Govt. may negotiate prices once the TA3 performer is determined.

E. Metrics

Symbiotic Design has several quantifiable objectives – Table 1 below summarizes the key program objectives and the anticipated quantitative progression through the program phases for illustration purposes. Proposers should develop and describe quantitative metrics specific to their approach aligning with the objectives listed below.

Table 1: Symbiotic Design Objectives

The program intends to quantify the technologies in different TAs as follows:

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 18

• (TA1.1) Design Space Construction – will be characterized with respect to the effectiveness of the mining techniques, and the desired measure is the number of components x number of options, with an ability to scale to 1000’s of components with 5- 10 (non-trivially distinct) options mined per component

• (TA1.2) Design Composition – will be quantified by the degree and accuracy of auto completion, with a target accuracy of 95% in Phase 3

• (TA1.3) Design Space Exploration – will be assessed with respect to the number of domains simultaneously explored, as well as the number of designs explored, and the target is to co-design across four domains, and explore millions of (non-trivially distinct) designs.

• (TA2) Symbiotic Exchange – will be assessed quantitatively as well as qualitatively. The quantitative evaluation will be with respect to the human effort required in the model construction and composition efforts, with a target of 90% reduction in human effort. The qualitative evaluation will be done using comparative evaluations. The TA3 performer will develop the protocol for evaluation.

Proposals should specifically list anticipated technical and programmatic risks and describe associated mitigation strategies.

F. Deliverables TA1 and TA2 performers will deliver appropriate code and documentation to the TA3 performer that they are working with at regular intervals and on a schedule consistent with the delivery of a working system one month before each demonstration meeting. The TA3 performer will deliver the seed designs, the design corpus, and all material needed for the challenge problem to the TA1 and TA2 performers that they are working with at regular intervals and on a schedule consistent with the delivery of a working system one month before each demonstration meeting. It is expected that the TA3 performers will be in close collaboration with the TA1 and TA2 performers at least one (1) month before each demonstration meeting to ensure program deliverables are met.

In addition, all performers will be required to provide the following deliverables:

• Technical papers covering work funded by Symbiotic Design;

• Source code, build scripts and any toolchains required to compile code, Algorithm and

Interface Description Document, user guides, other necessary data, and documentation, assumptions and limitations for all software developed under this program;

• Slide Presentations. Annotated slide presentations shall be submitted within one month after the program kick-off meeting and after each program event (program reviews, PI meetings, and technical interchange meetings);

• Quarterly Progress Reports. A quarterly progress report describing technical progress made, resources expended, major risks, planned activities, trip summaries, changes to key personnel, and any potential issues or problem areas that require the attention of the Government team shall be provided within 15 days after the end of each quarter;

• Monthly Progress Reports. A monthly progress report in the form of a PowerPoint document describing technical progress, planned activities for next month, any technical, financial, and programmatic issue shall be provided and presented in a teleconference with DARPA;

• Monthly financial status reports;

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 19

• A final phase report after each program phase. The final phase report shall concisely summarize the effort conducted within that phase; and

• Final Technical Report.

G. Intellectual Property Symbiotic Design will emphasize creating and leveraging open source technology. Intellectual property rights asserted by proposers are strongly encouraged to be aligned with open source regimes. Proposals entailing the use of existing non-public design corpora should discuss approaches to enabling the broadest possible distribution of the resulting Symbiotic Design technologies. See Section VI.B.1 for more details.

II. Award Information

A. Awards

Multiple awards are anticipated. The level of funding for individual awards made under this solicitation has not been predetermined and will depend on the quality of the proposals received and the availability of funds. Awards will be made to proposers whose proposals are determined to be the most advantageous to the Government, all factors considered, including the potential contributions of the proposed work, overall funding strategy, and availability of funding. See Section V for further information.

The Government reserves the right to:

− select for negotiation all, some, one, or none of the proposals received in response to this solicitation;

− make awards without discussions with proposers;

− conduct discussions with proposers if it is later determined to be necessary;

− segregate portions of resulting awards into pre-priced options;

− accept proposals in their entirety or to select only portions of proposals for award;

− fund proposals in increments and/or with options for continued work at the end of one or more phases;

− request additional documentation once the award instrument has been determined (e.g., representations and certifications); and − remove proposers from award consideration should the parties fail to reach agreement on award terms within a reasonable time or the proposer fails to provide requested additional information in a timely manner.

Proposals selected for award negotiation may result in a procurement contract, or Other Transaction (OT) depending upon the nature of the work proposed, the required degree of interaction between parties, and other factors. Grants will NOT be awarded under this program.

Proposers looking for innovative, commercial-like contractual arrangements are encouraged to consider requesting Other Transactions. To understand the flexibility and options associated with Other Transactions, consult http://www.darpa.mil/work-with-us/contract-management#OtherTransactions.

In accordance with 10 U.S.C. § 2371b(f), the Government may award a follow-on production contract or Other Transaction (OT) for any OT awarded under this BAA if: (1) that participant in http://www.darpa.mil/work-with-us/contract-management#OtherTransactions

HR001119S0083 SYMBIOTIC DESIGN FOR CYBER PHYSICAL SYSTEMS 20

the OT, or a recognized successor in interest to the OT, successfully completed the entire prototype project provided for in the OT, as modified; and (2) the OT provides for the award of a follow-on production contract or OT to the participant, or a recognized successor in interest to the OT.

In all cases, the Government contracting officer shall have sole discretion to select award instrument type, regardless of instrument type proposed, and to negotiate all instrument terms and conditions with selectees. DARPA will apply publication or other restrictions, as necessary, if it determines that the research resulting from the proposed effort will present a high likelihood of disclosing performance characteristics of military systems or manufacturing technologies that are unique and critical to defense. Any award resulting from such a determination will include a requirement for DARPA permission before publishing any information or results on the program. For more information on publication restrictions, see the section below on Fundamental Research.

B. Fundamental Research

It is DoD policy that the publication of products of fundamental research will remain unrestricted to the maximum extent possible. National Security Decision Directive (NSDD) 189 defines fundamental research as follows:

‘Fundamental research’ means basic and applied research in science and engineering, the results of which ordinarily are published and shared broadly within the scientific community, as distinguished from proprietary research and from industrial development, design, production, and product utilization, the results of which ordinarily are restricted for proprietary or national security reasons.

As of the date of publication of this BAA, the Government expects that program goals as described herein may be met by proposed efforts for fundamental research and…