HR001117S0003.pdf

Broad Agency Announcement Synergistic Discovery and Design (SD2)

HR001117S0003

November 22, 2016

Defense Advanced Research Projects Agency Information Innovation Office 675 North Randolph Street Arlington, VA 22203-2114

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 2

Table of Contents

I. Funding Opportunity Description

II. Award Information

A. Awards

B. Fundamental Research

III. Eligibility Information

A. Eligible Applicants

B. Organizational Conflicts of Interest

C. Cost Sharing/Matching

D. Other Eligibility Requirements

IV. Application and Submission Information

A. Address to Request Application Package

B. Content and Form of Application Submission

C. Submission Dates and Times

D. Funding Restrictions

E. Other Submission Requirements

V. Application Review Information

A. Evaluation Criteria

B. Review and Selection Process

VI. Award Administration Information

A. Selection Notices

B. Administrative and National Policy Requirements

C. Reporting

VII. Agency Contacts

VIII. Other Information

A. Frequently Asked Questions (FAQs)

B. Collaborative Efforts/Teaming

C. Proposers Day

D. Submission Checklist

E. Associate Contractor Agreement Clause (ACA)

F. Challenge Problem Examples

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 3

PART I: OVERVIEW INFORMATION

Federal Agency Name: Defense Advanced Research Projects Agency (DARPA), Information Innovation Office (I2O)

Funding Opportunity Title: Synergistic Discovery and Design (SD2)

Announcement Type: Initial Announcement

Funding Opportunity Number: HR001117S0003

Catalog of Federal Domestic Assistance Numbers (CFDA): Not Applicable

Dates o Posting Date: November 22, 2016 o Abstract Due Date: December 13, 2016, 12:00 noon (ET) o Proposal Due Date: February 14, 2017, 12:00 noon (ET) o Proposers Day: November 10, 2016

Anticipated Individual Awards: Multiple awards anticipated

Types of Instruments that May be Awarded: Procurement contracts or Other Transactions

Agency Contacts o Technical POC: Dr. Jennifer Roberts, Program Manager, DARPA/I2O o BAA Email: SD2@darpa.mil o BAA Mailing Address:

DARPA/I2O

ATTN: HR001117S0003

675 North Randolph Street Arlington, VA 22203-2114 o I2O Solicitation Website: http://www.darpa.mil/work-with-us/opportunities mailto:SD2@darpa.mil http://www.darpa.mil/work-with-us/opportunities

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 4

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

DARPA is soliciting innovative research proposals in the area of data-driven techniques and tools to enable scientific discovery and robust design. Proposed research should investigate innovative, revolutionary approaches to robust design in domains that lack complete models.

Specifically excluded is research that primarily results in incremental improvements to the existing state of practice.

This Broad Agency Announcement (BAA) is being issued, and any resultant selection will be made, using procedures under Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016.

Proposals received as a result of this BAA shall be evaluated in accordance with evaluation criteria specified herein through a scientific review process.

DARPA BAAs are posted on the Federal Business Opportunities (FBO) website (https://www.fbo.gov/).

The following information is for those wishing to respond to this BAA.

Introduction/Background

The Synergistic Discovery and Design (SD2) program aims to develop data-driven methods to accelerate scientific discovery and robust design in domains that lack complete models.

Engineers regularly use high-fidelity simulations to create robust designs in complex domains such as aeronautics, automobiles, and integrated circuits. In contrast, robust design remains elusive in domains such as synthetic biology, neuro-computation, and polymer chemistry due to the lack of high-fidelity models. SD2 will develop tools to enable robust design despite the lack of complete scientific models.

Examples of complex systems where inventors lack complete scientific models to support their design efforts include biological systems that have millions of protein-metabolite interactions, neuro-processes that require computations across billions of neurons, and advanced materials influenced by millions of monomer-protein combinations. These systems are part of domains that exhibit millions of unpredictable, interacting components for which robust models do not exist, and internal states are often only partially observable. In such domains, small perturbations in the environment can lead to unexpected design failures, and the number of engineering variables required to characterize stable operational envelopes remains unknown.

While domain experts remain geographically dispersed, they collectively analyze hundreds of terabytes of data to build models and refine designs. However, manually-intensive analysis of small datasets remains inefficient and often yields unreproducible results. In response, researchers have begun to outsource high-throughput experiments to automated labs and randomly search constrained parameter spaces for robust designs. However, these random search-based approaches work best with small parameter spaces and provide no insight into why some designs succeed and others fail. SD2 aims to develop data-driven methods to automatically discover models and refine designs in parallel at scale.

https://www.fbo.gov/

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 5

Program Description

SD2 will address the problem of design in domains that lack complete models by discovering models and refining designs via methods that extract information from data at petabyte scale. To ensure realism, challenge problems drawn from cutting edge domains will drive the development of SD2 methods.

Challenge Problems Program wide evaluations of SD2 methods will involve a series of real world design challenges that today’s scientific community does not yet know how to solve. The intent is to develop data-driven methods that can generate new scientific discoveries and design new capabilities that are not currently found in literature and are outside the limits of current science. SD2 methods will be evaluated based on their ability to discover new biological, chemical, and/or physical properties of complex systems at the cutting edge of research. SD2 will conduct experiments over the course of the program to test hypotheses and designs to determine whether new capabilities have been created. Program methods will succeed if they can solve the challenge problems at a rate faster than current publication cycles or decrease the expected time to convert an academic finding into a robust, industrial design or process.

Initially, SD2 will use challenge problems from synthetic biology for program wide evaluation.

Synthetic biology provides a compelling driving application domain for the following reasons:

(1) full characterization of the underlying biology requires complex mechanistic formalisms at a scale that defies manual discovery; (2) advances in the last 20 years have enabled scientists to more efficiently modify organisms in more complex and targeted ways; (3) terabytes of high quality synthetic biology data can be generated in a few days; and (4) design of synthetic biology systems remains a very laborious process that typically involves heuristic knowledge, brute force, and trial and error approaches.

Novel, high-impact challenge problems from alternative application domains are also in scope.

SD2 will select challenge problems from alternative application domains based on the ability to collect high fidelity experimental data that can test the generality of the data-centric methods.

Domains of interest include, but are not limited to, chemistry, material science, and neuro-computation. High impact problems that facilitate open data sharing and collaboration free of proprietary restrictions are encouraged.

Program Evaluation Program wide evaluation will be organized around quarterly demonstration efforts based on challenge problems. The challenge problems will drive performers to algorithmically generate hypotheses and designs that can be experimentally tested within days, weeks, or months.

Methods developed during the program should reduce the human effort required to achieve challenge goals.

At regular intervals, SD2 performers who are subject-matter experts will propose increasingly complex challenge problems based on lessons learned from experimental successes and failures.

Notional design challenge problems may include the ability to design a 30 component circuit for nuclear waste absorption (in synthetic biology) or design a polymer that constricts in response to contact with a toxin (in polymer chemistry). Discovery challenges will reduce uncertainty associated with unpredictable experimental results. For example, discovery challenges might include the capability to identify environmental parameters that enables an unstable biological

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 6

circuit to operate as long as possible or diagnose root cause for experimental failure. Challenge problems will be screened to ensure that they:

• Test the ability to accelerate robust design in domains that lack models;

• Lead to discovery and design hypotheses that can be experimentally tested in days, weeks, or months;

• Do not pose Intellectual Property (IP) or data sharing restrictions; and

• Explore topics relevant to DoD interests, such as reproducibility and robustness.

Program Structure

Model discovery is a primary research focus for SD2, as accurate models can accelerate design in domains that lack high fidelity simulations. To facilitate model discovery, SD2 plans to advance computational analysis of raw experimental data at petabyte scale. The program will virtualize experimental workflows in order to increase access to high-throughput experimental resources. High-throughput experiments will provide large increments of high quality data from multiple labs. This data will flow into an SD2 analysis hub that serves as a data-sharing ecosystem for scalable, automated discovery and design. Novel discovery algorithms generated by the SD2 program will automatically process experimental data with the objective of detecting unexpected findings and refining system models. SD2-developed design algorithms will use the refined models to generate, test, and validate system designs. SD2 techniques and tools will be evaluated using challenge problems drawn from domains such as synthetic biology, polymer chemistry, and neuro-computation.

Technical Areas The program structure includes five technical areas (TAs) that must work collaboratively to achieve program goals. SD2 encourages proposers to carefully review expectations for all five TAs and examine the “Demonstration Efforts” section (see page 8) in order to understand context for the TA to which they plan to propose. Short descriptions for each TA appear immediately below and detailed descriptions appear later in this document.

TA1 (Data-Centric Scientific Discovery): Develop computational workflows and algorithms to automatically detect significant variations between expected versus observed outcomes (hereafter termed “experimental surprise”); extract patterns, outliers, and design constraints from experimental data at scale; and use patterns to refine models.

TA2 (Design in the Context of Uncertainty): Extend engineering and planning formalisms to enable automated refinement of intermediate designs.

TA3 (Hypothesis and Design Evaluation): Experimentally evaluate the designs and hypothesized system knowledge generated by TA1 and TA2 in a manner that tests for reproducibility and robustness.

TA4 (Data and Analysis Hub): Extend open source tools to virtualize access to computational resources, data, and results in a manner that links multiple labs and research communities.

TA5 (Challenge Problem Integrator): Work with domain experts throughout the program to establish quarterly challenge problems to drive development of automated, data-driven methods that accelerate complex system design at scale.

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 7

Program Workflow Figure 1 shows the workflow SD2 will use to develop new methods and to evaluate the resulting techniques and tools on real world challenge problems. Figure 2 specifies the input/output interactions between each program TA.

Figure 1: SD2 Program Structure and Workflow

TA1 TA2 TA3 TA4 TA5

TA1

Design constraints, models, and levels of uncertainty

Hypotheses and notional experiments during Phase 1

Discovery algorithms/workflows

Insights extracted from data to propose challenge problems

TA2

Information requirements for design

Proposed experimental plan to test designs and hypotheses

Design algorithms Insights extracted from data to propose challenge problems

TA3 Data to test and demonstrate methods

Data to test and demonstrate methods, API for submitting experimental requests

Experimental data Subject matter expertise to propose challenge problems

TA4

Infrastructure to demonstrate methods, APIs for running methods during data ingest

Infrastructure to demonstrate methods; APIs for running methods during data ingest

Hardware to facilitate data transfers, Data formatting requests

Insights extracted from trend analysis to propose challenge problems

TA5 Delivery of discovery challenge problems

Delivery of design challenge problems

Prioritization across experiments proposed by TA1 and TA2

Pr od uc er

Receiver

Figure 2: TA Interaction Matrix

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 8

Note: All application program interfaces (APIs) should be rapidly prototyped and performers should regularly collect feedback from the TAs using the API.

Collaborative Program Structure DARPA intends for SD2 to be a highly collaborative program in which all performers (both within and across TAs) will constructively interact with one another. To facilitate the open exchange of information, performers will have an Associate Contractor Agreement (ACA) clause included in their award. This agreement is between performers that specify requirements to share information, data, technical knowledge, expertise or resources. The ACA agreement is intended to ensure appropriate coordination and integration of work by SD2 performers. For more information, see Section VIII.E.

Program-wide evaluation will require significant collaboration across all TAs. Regardless of initial teaming arrangements, SD2 performers will be expected to collaborate closely with other performers within their TA, as well as with performers of other TAs of the program. All SD2 performers must work cooperatively with one another to develop, integrate, test, and evaluate SD2 capabilities. TA4 infrastructure will facilitate collaboration among performers. As such, all proposals must include plans for delivering all necessary software, data, and domain knowledge to the TA4 performers. Similarly, experts from all TAs should plan to share experimental outcomes, lessons learned, and challenge problem ideas with one another and with the TA5 performer. Full interoperability of the prototype tools will not be required.

To promote collaboration among all performers, SD2 will hold quarterly working PI meetings, requiring the attendance of representatives from all TAs. The TA5 performer is responsible for organizing the technical agenda for these meetings in coordination with Government representatives. The intent of the working PI meetings is to facilitate collaboration among all SD2 performers, coordinate data management efforts, exchange insights gained from validation experiments, evaluate the effectiveness of discovery and design algorithms, refine challenge problems, and prioritize experiments for the next quarter.

During each quarterly working PI meeting, SD2 performers with experience in the selected application domain(s) will work together to choose compelling challenge problems for the following quarter. Each challenge problem should attempt to push the bounds on data-driven methods for model discovery and design refinement. The TA5 performer will act as a facilitator among the domain experts across the program to ensure the program takes advantage of all relevant expertise. Upon the TA5 performer finalizing the quarterly design challenge problem in coordination with the Government, TA2 performers must evaluate the scientific knowledge contained within the Data and Analysis Hub for relevance to the design challenge. Any gaps in knowledge will form the basis for data-centric, discovery, challenge problems for TA1. It is anticipated that by the end of each PI meeting, program performers (all TAs) should use lessons learned to agree on the next challenge problem(s) and agree on validation experiments for the next quarter. TA3 will execute the validation experiments prior to the next quarterly PI meeting.

Then TA1 and TA2 will evaluate the experimental data in the following quarter. Figure 3 shows the anticipated timeline for the quarterly challenge problem cycle.

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 9

Figure 3: Notional Quarterly Challenge Problem Timeline

Demonstration Efforts In response to challenge problems, SD2 performers will demonstrate methods on a steadily growing repository of experimental data. Each quarter, performers will present novel findings at the scheduled PI meeting. Proposers should plan to dynamically prioritize method development based on lessons learned during challenge problem demonstrations and feedback received through validation experiments. Proposals that predominantly focus on analysis of non-program datasets will be considered non-responsive.

Section VIII.F gives examples of design challenge problems in the field of synthetic biology that are appropriate for SD2. Novel, high-impact, challenge problems from alternative application domains that require relatively small data collects may also be proposed.

The following sections outline each technical area in more detail.

Technical Area 1: Data-Centric Scientific Discovery TA1 performers must develop computational methods that convert experimental data into scientific knowledge that design algorithms can use for computation. TA1 algorithms must analyze experimental data, detect experimental surprise, diagnose failure, extract patterns, and use those patterns to refine system models. SD2 aims to reduce the amount of human intervention required for discovery tasks by creating methods that automate routine, mundane operations. Such automation will refocus human efforts on interesting scientific questions.

Proposers are encouraged to develop a reconfigurable set of tools that can discover scientific models from arbitrary experimental datasets. The reconfigurable tools should extract qualitative and quantitative insights from experimental data in a manner that incrementally builds models of increasing complexity. Ostensibly, the progression from data through qualitative observations to quantitative models involves four computational steps:

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 10

i) Discover categories of engineering variables (e.g., classes of proteins or genes with similar binding characteristics);

ii) Qualitatively characterize relationships between categories (e.g., protein category A inhibits gene category B);

iii) Identify stable regions of operation (e.g., the inhibitory effect occurs only at room temperature when the cell is not replicating); and

iv) Quantify relationships between engineering variables (e.g., the inhibitory effect is twice as strong when the amount of sugar in the environment doubles).

Proposers must describe a detailed technical approach that addresses how all four of these computational steps could be performed on arbitrary datasets using a reconfigurable set of tools.

Approaches that combine multiple machine learning or data processing methods are in scope, as are approaches that expand the representational expressivity of a single modeling formalism.

Model formalisms of interest include but are not limited to causal models, generative models, discriminative models, finite state machines, categorical grammars, dynamical system models, logical models, and graphical models.

Proposers should anticipate that models can periodically be validated via new experiments.

Methods that incrementally increase model complexity at a rate commensurate with empirical evidence are of interest. Proposers are encouraged to address how they plan to identify suitable modeling formalisms, compare the relative validity of models both within and across formalisms, and provide a principled approach for moving from qualitative to quantitative formalisms based on experimental evidence.

Proposers should explain how their reconfigurable tools will reduce the human intervention required for a variety of discovery tasks traditionally performed by scientists. Discovery tasks of interest include, but are not limited to, the following objectives:

Detect experimental surprise (i.e., significant variations between expected versus observed outcomes);

Diagnose root causes for experimental failure;

Hypothesize latent variables that account for experimental observations, Build and assess models;

Construct operating envelopes to identify stable system behaviors that might support robust design; and Discover new scientific concepts or breakthroughs.

Proposers should highlight how their approach will address several such discovery tasks.

Ultimately, SD2 hopes to enable analysis of data from hundreds of thousands experiments with minimal human intervention. Computational methods that extract information primarily from the experimental data generated by TA3 are preferred. Methods that use literature or subject matter expertise to constrain model types or parameters are also considered in scope. However, approaches that rely heavily on human extraction from literature may be considered non-responsive.

All TA1 computational methods must scale to operate over relevant subsets of SD2’s steadily growing repository of experimental data as described in TA3. TA1 proposals must describe the

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 11

scale at which baseline implementations will operate and assess the scale required to demonstrate methods on the full data repository. If needed, proposers must provide a technical plan to recast algorithms to a scale commensurate with the SD2 experimental data repository.

TA1 proposals must clearly delineate the types of scientific hypotheses each proposed computational method can extract. All TA1 computational methods must either formulate a testable hypothesis or provide information in a form that enables a subject matter expert to form a testable hypothesis. Scientific hypotheses and models generated by TA1 algorithms will be experimentally validated by TA3 performers. TA1 proposals must explain what information will be provided to TA2 and TA3 to formulate validation experiments.

All TA1 outputs and workflows must be shared with other SD2 performers via the Data and Analysis Hub. Proposers should plan to fine-tune their technical approach based on lessons learned during challenge problem demonstrations on program provided datasets. For example, TA1 performers should dynamically adjust a model’s level of abstraction based on challenge problems and data. An ideal level of abstraction would provide enough detail to accelerate program wide progress toward design challenges and would limit details that did not substantially improve design quality.

The experimental data used to develop TA1 algorithms will be generated by TA3 or provided by the Government via TA4 infrastructure. Proposals that predominantly focus on analysis of non-program datasets will be considered non-responsive. TA1 performers must use TA4 infrastructure for algorithm development whenever appropriate.

Technical Area 2: Design in the Context of Uncertainty TA2 performers must develop design algorithms that provide novel design capabilities in domains with incomplete models. Design algorithms will be used to address domain-specific design challenges such as: design a 30 component circuit for nuclear waste absorption (in synthetic biology) or design a polymer that constricts in response to toxin (in polymer chemistry). The TA2 design algorithms will consist of two classes: (1) algorithms for targeted design of system components, and (2) algorithms for experimental planning. All TA2 and joint TA1/TA2 proposals must address experimental planning. TA2-only proposals must also address targeted design of system components. TA1/TA2 proposals may optionally address targeted design of system components. The experimental data used to develop TA2 algorithms will be generated by TA3 or provided by the Government via TA4 infrastructure.

TA2-only proposals must specify the intended application domain for targeted design and outline a notional design challenge. SD2 is open to any design challenge within synthetic biology that can be tested via data generated by traditional synthetic biology data collects (e.g., omics, HPLC, flow cytometry, etc.). SD2 is also open to challenges in other applications as long as the TA2 proposal references a notional TA3 team that can provide experimental data to evaluate designs.

Proposers interested in teaming are encouraged to visit the SD2 teaming website at:

https://www.schafertmd.com/darpa/i2o/sd2/pd.

TA2-only proposals must outline a detailed technical approach to achieve targeted design of system components. Targeted design algorithms should enable users to specify a functionality requirement appropriate for the application domain and receive designs that have the potential to attain desired functionality. For example, a component design algorithm might propose DNA

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 12

constructs that enable a single-celled organism to fluoresce in response to a toxin. Design approaches that require significant manual extraction of information from literature, hand-coded heuristic knowledge, and manually coded design rules are out of scope.

Proposers must describe the required inputs and resulting outputs of the algorithm. For example, DNA design capabilities for synthetic biology might require algorithm inputs such as a molecule to be detected, type and duration of detection signal, and required operating conditions (e.g., temperature, suspended vs. plated, media composition, etc). Resulting outputs might include biological circuit design specifications (e.g., DNA construct, circuit logic, circuit inputs and outputs, host organism, and the range of environmental conditions across which the circuit should function).

TA2-only proposals must also include a sound technical plan to automatically and dynamically incorporate design constraints from experimental data and TA1 models into the targeted-design algorithms. Approaches should incorporate increasingly complex information from TA1 throughout the program. TA2-only proposals must specify assumptions about TA1 outputs that would provide added value to the TA2 approach. Knowledge representations of interest might include design rules or constraints, design grammars, and/or empirical distributions.

All TA2 proposals must include a detailed technical approach that specifies how to generate prioritized experimental plans and algorithms that derive plans for testing component designs and TA1 hypotheses. TA2 proposals should also address how to:

Prioritize experiments based on the anticipated information theoretic value;

Assess utility of running an experiment or sequence of experiments;

Minimize the number of experiments required to develop and validate robust, reproducible designs that function across labs and across pre-specified ranges of environmental contexts;

Prioritize the measurements and number of samples required for a particular experiment;

Specify which experiments to run in one laboratory location instead of multiple laboratory locations; and Maximize information that can be gained from experiments while minimizing cost.

Experimental planning algorithms must be domain agnostic. TA2 proposals should describe approaches that require minimal reconfigurations to perform experimental planning functions in multiple domains. Inputs to the experimental planning algorithms might include available actions (e.g. experimental measurements) from TA3, component designs or hypotheses from TA1 and TA2 algorithms, costs associated with actions, available instrumentation, and consumables. The experimental planning algorithm should output a list of prioritized experiments that minimize the experimental overhead required to converge on a successful design. The prioritized list should specify details required to execute the experiments, such as the types of consumables, instrumentation, and experimental workflows.

TA2 proposals must describe how component design and experimental planning algorithms will decrease the percentage of mundane tasks that require human intervention. TA2 proposals should also describe the level of domain expertise required to use the proposed design tools.

Approaches that substantially reduce the level of domain expertise required to generate successful designs are of particular interest.

TA2 performers must use TA4 infrastructure for algorithm development whenever appropriate.

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 13

Technical Area 3: Hypothesis and Design Evaluation TA3 performers must generate experimental data throughout all phases of the program to validate TA1 hypotheses and evaluate TA2 designs. TA3 performers must provide computer readable application program interfaces (APIs) that enable both subject matter experts and computer algorithms to submit experiment requests. Ideally, automated workflows will remotely log experiment and contextual data in a manner that facilitates data comparison between geospatially distributed laboratories.

TA3 proposals must specify an intended application domain, describe data collection capabilities, and outline at least three relevant challenge areas. For each challenge area, proposers should identify a series of notional design and discovery challenges that could drive TA1 and TA2 development. Proposals should connect challenge problems with notional experiments that would provide relevant data. For each challenge, TA3 proposers should also explain the impact that a novel solution would have within the research community or the Department of Defense (DoD). More information about the types of challenge problems of interest to SD2 can be found in the “Challenge Problems” section (see page 5) and “Demonstration Efforts” section (see page 8).

SD2 will use synthetic biology as an initial application domain. However, TA3 proposals are not required to specify synthetic biology as the intended application domain. Novel, high-impact challenge problems from alternative application domains are also in scope. Alternative domains of interest to SD2 include polymer chemistry, synthetic chemistry, and neuro-computation. All TA3 proposals must provide experimental capabilities that support rapid experimental cycles (days, weeks, or months), open data sharing, computer readable experimental workflows, and cross lab data calibration. TA3 proposals using microbial, preclinical (animal), cell-culture, and cell-free experiments are in scope. However, TA3 proposals that involve human subjects research (HSR) are not in scope.

TA3 proposals must outline a sound technical plan for providing access to experimental resources via an API. Competitive teams will provide detailed, computer-readable, workflow languages that precisely describe experimental protocols, track environmental sensor readings, support supply chain management, and facilitate cross-lab comparisons. Proposals should address how the data collection techniques will facilitate data calibration and comparisons across geo-spatially distributed labs.

By the end of the program, integrated TA2/TA3 capabilities will enable experimental planning algorithms to submit experiment requests. Open source APIs are desired to provide the interface between TA3 experimental capabilities and TA2 algorithms. As such, TA3 proposals must include a description of APIs with notional lists of experimental commands, parameters, and associated costs. TA3 proposals should address how to incorporate novel instruments and experimental protocols in a manner that should not require recompilation of TA2 code.

All data generated by TA3 will be open source, and all data must be transferred to TA4 performers for ingestion into the Data and Analysis Hub. Proposers must describe how they plan to support data transfers to the Data and Analysis Hub (TA4).

TA3 proposals must include sound, detailed technical descriptions of domain-specific data collection capabilities. TA3 proposals must describe the level of automation for each experimental capability. High levels of automation are preferred. SD2 has strong interest in

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 14

unique, high-throughput, experimentation capabilities combined with reproducible, fine-grained, experimental capabilities. Fine-grained capabilities may rely on low- or mid-volume manual protocols, but they must provide high-value perspectives that can be calibrated against larger data collects. Strong proposals in TA3 will provide complementary capabilities that include both high-throughput measurements, as well as a broad range of fine-grained, experimental protocols.

Proposers may highlight existing datasets that will be augmented with experiments conducted throughout the program.

TA3 performers must have access to equipment that enables data collection at appropriate volume and variety for the specified application domain. Proposals specifying synthetic biology data collects must satisfy the following criteria:

Labs physically located in at least 3 geographically distinct areas;

Equipment and staff to perform experiments with different types of single-celled organisms and cell lines;

High-throughput robotics equipment and staff to collect high fidelity omics and biophysical data collectively at a rate of 100 GB to 1 TB per day;

Breadth of experimental capabilities to address a large variety of design and discovery challenges;

Sensor instrumentation to collect parallelized environmental and cellular contextual data;

and Computer readable workflow language to enable cross lab comparison.

TA3 proposals specifying alternative application domains must satisfy the following criteria:

Instrumentation that produce a breadth of data on the order of megabytes (MB) to 100s GBs per day;

Sensor instrumentation to collect parallelized environmental and contextual data;

Computer readable protocol language to enable cross lab comparison; and Novel, high-impact data collects.

All TA3 proposals must include a table describing all types of data collect, associated instrumentation, rate of sample preparation, rate of data generation, data format, amount of data generated per sample, and resources compatible with the data collect. The table should be constructed using the following template:

Type of Data Collect Instrumentation

Rate of sample preparation

Maximum rate of data generation

Format of data, typical file size per sample

Organisms

Proteomics (Protein peak heights)

(model #, Vendor) 100/day 1 TB/day .txt file, 1GB Bacteria Yeast

QPCR

(Cycle Threshold values)

(model #, Vendor) 1000/day 50GB/day .xml file, 2GB Bacteria Yeast Mammalian

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 15

Note: The above table was written assuming synthetic biology as the application domain. TA3 proposals specifying alternative application domains must include a table with similar level of detail.

Within synthetic biology, high-throughput instrumentation of interest includes, but is not limited to: proteomics, mass spectrometry-based techniques, live cell multiplex assay, flow cytometry, and next generation sequencing. TA3 proposals to alternative application domains must have access to equipment that enables a similar breadth and depth of reproducible experimental capabilities.

In addition to instrumentation, TA3 proposals must list other types of experimental resources.

For instance, synthetic biology proposals should include organisms and/or cell lines supported by the experimental facility as well as special consumables and novel capabilities. For example, proposals should specify if the proposer can synthesize DNA constructs for the entire program.

All TA3 proposals should specify the approved safety levels of their laboratories. Organisms of interest include, but are not limited to: Prokaryotes such as Escherichia coli, Bacillus subtilis, Salmonella typhimurium and Eukaryotes such as Saccharomyces cerevisiae and Pichia pastoris.

Novel and less well characterized organisms such as extremophiles are also of interest.

A single TA3 performer should satisfy as many of the above criteria for laboratory capabilities as possible within the specified application domain. Teaming between TA3 performers is encouraged. SD2 has provided a teaming website to facilitate coordination of teams (https://www.schafertmd.com/darpa/i2o/sd2/pd/). To facilitate the open exchange of information between multiple laboratories, TA3 selected proposals will contain an ACA clause in any FAR based award (and similar data collaboration language in other awards) and ACAs will need to be in place before the first program kick-off meeting. For more information, please see section

VIII.E.

Technical Area 4: Data and Analysis Hub TA4 performers must establish and maintain a data sharing ecosystem that facilitates rapid data ingest, collaborative data analysis, and computational research. The ecosystem will require purchase or rental of cyber-infrastructure (i.e., computational hardware, storage services, and networking), installation and configuration of software, as well as the development of tools for data management, computation, and collaboration. TA4 is responsible for ingesting and pre-processing scripts, baseline visualization scripts, baseline compute for all TA1-4 data processing, and hardware for TA3/TA4 data transfers (see Figure 4).

https://www.schafertmd.com/darpa/i2o/sd2/pd/

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 16

Figure 4: TA4 responsibilities associated with the notional data processing pipeline

TA4 performers must provide infrastructure that enables an alternative approach to scientific research. The TA4 team must flexibly incorporate high-impact capabilities into the data and analysis hub based on lessons learned throughout the program. Strong proposals will describe a compelling vision beyond the basic requirements outlined in this section.

TA4 proposals must include a sound, detailed, technical plan that describes hardware, user-management, data management, analytic environments, integration, collaboration tools, and maintenance. Proposals should clearly identify existing commercial or open source tools that will be leveraged over the course of the program. Proposals should also clearly delineate which technical requirements necessitate novel extensions to existing tools. Sections describing novel extensions should include a clear technical plan that first outlines current capabilities and then explains how tools will be extended, adapted, or developed to achieve TA4 technical objectives.

TA4 proposals must describe a plan for provisioning hardware that supports data ingest at an average rate of 1 terabyte (TB) per day over four years, supports central processing units (CPU) and graphic processing units (GPU) computations, enables performers in all TAs to perform collaborative data analysis, dynamically adjusts to program-wide storage and compute needs, and facilitates long-term archival storage. TA4 will be responsible for provisioning storage and compute for the entire program as well as for development, but not execution, of a plan for data preservation and curation of the results beyond the end of the period of performance. During the program period of performance, cyber-infrastructure provided should accommodate active archival storage for all raw data and fast access to data required for computational research.

Proposers should define reasonable assumptions about aggregate storage and compute requirements across TA1-TA3 and use those assumptions as the basis for cost estimates.

To reduce cost, the hardware plan may include options to incrementally augment the core system with additional storage and compute at regular intervals throughout the program. Approaches of interest include, but are not limited to, incremental hardware purchase, hardware as a service, or infrastructure as a service. Proposers should clearly explain whether they plan to use public, private, or hybrid infrastructure. The cyber-infrastructure must facilitate high speed connections for users distributed across the country, enable low latency data processing from distributed research locations, and support data transfers from multiple locations. Cyber-infrastructure may support either streaming ingest from multiple TA3 labs or batch ingests at regular intervals. TA4

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 17

proposals must include a clear plan that describes the hardware used to facilitate information transfer.

TA4 proposals must also include a user-management plan that addresses access to, and provisioning of, storage and compute. TA4 is responsible for baseline compute and storage for all TAs. If the proposed hardware plan involves private infrastructure, the user-management plan should address access. If the proposed hardware plan involves a public cloud provider, the user-management plan should describe a technical approach that ensures novice users do not accidentally run jobs with unnecessarily expensive compute. Ideally, the user-management approach will have automated mechanisms to track jobs and ensure that scripts are first tested for scalability on small data samples before running on large data tables. Solutions that adjust baseline compute thresholds based on script history or user expertise are encouraged. Solutions that automatically pause or kill jobs that exceed projected compute profiles are also encouraged.

For public cloud solutions, the user-management plan should also address how to provide non-program participants with access to data but not program-funded compute.

TA4 proposals must also contain a data-management plan, which should explain in detail how proposers will address the following technical objectives:

Support data ingest, indexing, search, and filtering operations over raw experimental data, environmental sensor data, and experimental workflow data (see TA3 for additional information);

Support data ingest and pre-processing at an average rate of 1 TB per day;

Support robust ingest processes for common data formats;

Provide a rapid ingest process for data with novel formats;

Store data in manner that facilitates exploration and algorithmic processing by TA1-

TA4 performers;

Track data provenance for raw and pre-processed data;

Provide APIs and/or conventions for tracking the provenance of outputs generated by

TA1, TA2, and TA3 algorithms;

Adaptively archive or re-index data based on usage patterns so that common operations occur quickly and infrequently used data is archived or flagged for deletion; and

Retain all raw data in a manner that would support infrequent re-processing as needed.

Of particular interest are approaches that track usage statistics and algorithmically propose adaptive data retention policies based on usage patterns, data provenance, and data type. Ideally, raw data would be archived, while processed data would be retained, archived, or flagged for deletion based on usage and provenance. Data should be stored in a manner that makes common operations fast and uncommon operations possible. Proposers should plan to take advantage of usage statistics to distinguish between common and uncommon operations. Methods that promote syntactic and semantic interoperability are encouraged. Approaches that provide flexible support for new data types are encouraged, while approaches that rely on brittle schema are discouraged.

Proposers are encouraged to develop a data management plan with security controls for multi-tenant environments. Ideally, the technical approach will provide mechanisms to add security controls during years three and four of the SD2 program. Ideally, the security controls would

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 18

enable sensitive datasets to be shared, processed by TA1/TA2 algorithms, and explored only by approved users.

TA4 proposals must also include a technical plan to establish a flexible analysis environment that

(1) regularly incorporates open source tools as they become available, and (2) provides core data pre-processing, statistical analysis, and data visualization capabilities. Proposed analysis environments should address the following technical objectives:

Provide and demonstrate tools that support rapid exploration, sampling, and statistical analyses of novel datasets;

Automatically generate data summaries, trend analyses, and exploratory visualizations on newly loaded datasets;

Provide a library of just-in-time pre-processing methods that extract features from tables with raw experimental data (e.g. extract protein peaks from mass spectrometry data or perform BLAST alignment of genomic data);

Minimize the development time required to integrate pre-processing methods in novel domains;

Provision the analysis environment with novel open source research tools. Open source tools of interest include, but are not limited to, Spark, Spark Machine Learning library, DeepLearning4j, Caffe, R-Hadoop, and python;

Provide TA1 and TA2 with a library of open source data visualization tools that facilitate display and exploration of analytic results with common formats (e.g., time series, histograms, event sequences, and network data types); and

Provide logging tools and resource management visualizations to facilitate algorithm debugging within the data and analysis hub.

TA4 proposals must include an integration plan that outlines how the team will facilitate integration of raw data from TA3 performers and integration of TA1/TA2 algorithms into the data processing workflow. The integration plan should address the following technical objectives:

Establish a process for regular data ingest from TA3 labs at an average rate of 1TB per day;

Work with TA3 and TA1 to develop formatting conventions that facilitate rapid ingest of raw data from multiple labs;

Support periodic data transfers from Government provided data sources;

Ensure that the analysis environment contains open source research tools and visualization components commonly required by TA1, TA2, or TA3 performers;

Provide an integration framework for TA1 and TA2 algorithms;

Ensure that integration APIs track algorithm usage and data provenance;

Work with TA1 and TA2 to identify common pre-processing steps and workflows, as well as incorporate common operations into an automated data ingest pipeline;

Establish communication processes that highlight the status of TA4 products, such as data ingest tables, data pre-processing tables, integration of common operations, and updates to data visualizations;

Evaluate which scaling techniques work best across the program;

Establish best practices with regards to parallelization, scalability, data ingest, and data provenance;

Provide tutorials to TA1, TA2, and TA3 performers regarding best practices; and

HR001117S0003 SYNERGISTIC DISCOVERY AND DESIGN 19

Provide liaisons to other technical areas to facilitate data ingest, data processing, establishment of the analysis environment, and debugging of algorithms at scale.

Full integration of TA1 and TA2 algorithms is not required. However, TA4 must integrate methods that TA1 and TA2 performers consistently want to run at data ingest.

In addition, TA4 proposals must include a technical plan to demonstrate collaboration tools that enable researchers to share workflows and results. Proposed collaboration environments should address the following technical objectives:

Develop, extend, or identify tools that enable researchers to share workflows and results with collaborators who use different operating systems or different analysis environments;

Enable researchers to explore each other’s analytic workflows and results in a manner that does not require substantial data downloads or time-consuming installation of new software on desktop or laptop computers;

Track usage statistics for software methods and data tables; and Develop collaboration tools that highlight popular data tables, visualizations, analytic workflows, and results.

Whenever possible, proposers are encouraged to incorporate open source technologies, including but not limited to Vagrant, Docker, Jupyter Notebooks, HubZero, and iRODS.

TA4 proposals must also include a hardware and software maintenance plan. Maintenance plans should address how to efficiently perform software upgrades, respond to bugs, maintain software repositories, manage cloud configurations, and upgrade hardware as needed. Proposers should explain how the technical approach will minimize the manpower required for maintenance.

Proposers are encouraged to highlight best practices and refer to relevant tools, such as Puppet, Chef, or Salt.

TA4 proposals will be evaluated on the completeness, achievability, and feasibility of the technical plans for hardware, user-management, data management, analytic environment, integration, collaboration tools, and maintenance. Proposals will also be evaluated based on the novelty of any proposed extensions. Of particular interest are novel open source solutions that work across a variety of public and private infrastructures in a cost-effective way. Proposals with strong biological and experimental data processing techniques are encouraged. In addition, strong proposals will include a comprehensive approach for adaptive data management and code integration based on data provenance and usage statistics.