HR001119S0038.pdf

HR001119S0038 SAIL-ON 1

Broad Agency Announcement Science of Artificial Intelligence and Learning for

Open-world Novelty (SAIL-ON)

Defense Sciences Office

HR001119S0038

March 15, 2019

HR001119S0038 SAIL-ON 2

Table of Contents I. Funding Opportunity Description

A. Introduction B. Background C. Program Description/Scope D. Program Structure E. Technical Area Descriptions F. Schedule/Milestones G. Deliverables H. Government-furnished Property/Equipment/Information I. Other Program Objectives and Considerations

II. Award Information A. General Award Information B. Fundamental Research

III. Eligibility Information A. Eligible Applicants B. Organizational Conflicts of Interest C. Cost Sharing/Matching D. Other Eligibility Requirements

IV. Application and Submission Information A. Address to Request Application Package B. Content and Form of Application Submission C. Submission Dates and Times D. Funding Restrictions E. Other Submission Requirements

V. Application Review Information A. Evaluation Criteria B. Review and Selection Process C. Federal Awardee Performance and Integrity Information (FAPIIS)

VI. Award Administration Information A. Selection Notices B. Administrative and National Policy Requirements C. Reporting

VII. Agency Contacts VIII. Other Information

A. Frequently Asked Questions (FAQs) B. Proposers Day C. Sample Associate Contractor Agreement Text

BAA Attachments:

Attachment A: ABSTRACT SUMMARY SLIDE TEMPLATE Attachment B: ABSTRACT TEMPLATE Attachment C: PROPOSAL SUMMARY SLIDE TEMPLATE Attachment D: PROPOSAL TEMPLATE VOLUME 1 TECHNICAL & MANAGEMENT VOLUME Attachment E: PROPOSAL TEMPLATE VOLUME 2 COST VOLUME Attachment F: PROPOSAL TEMPLATE VOLUME 2 COST SUMMARY SPREADSHEET Attachment F-2: PROPOSAL TEMPLATE VOLUME 2 COST BREAKDOWN Attachment G: PROPOSAL TEMPLATE VOLUME 3 ADMINISTRATIVE & NATIONAL POLICY REQUIREMENTS VOLUME

HR001119S0038 SAIL-ON 3

PART I: OVERVIEW INFORMATION

Federal Agency Name: Defense Advanced Research Projects Agency (DARPA), Defense Sciences Office (DSO)

Funding Opportunity Title: Science of Artificial Intelligence and Learning for Open-world Novelty (SAIL-ON)

Announcement Type: Broad Agency Announcement

Funding Opportunity Number: HR001119S0038

Catalog of Federal Domestic Assistance (CFDA) Number(s): 12.910 Research and Technology Development

Dates (All times listed herein are Eastern Time.)

o Posting Date: March 15, 2019 o Proposers Day: March 5, 2019. See Section VIII.C.

o Abstract Due Date: April 2, 2019, 4:00 p.m.

o FAQ Submission Deadline: April 30, 2019, 4:00 p.m. See Section VIII.A.

o Full Proposal Due Date: May 10, 2019, 4:00 p.m.

Anticipated Individual Awards: DARPA anticipates multiple awards for Technical Area 1 and multiple awards for Technical Area 2. See Section II.A.

Types of Instruments that May be Awarded: Procurement contracts, cooperative agreements or other transactions.

Agency contacts o Technical POC: Ted Senator, Program Manager, DARPA/DSO o BAA Email: SAILON@darpa.mil o BAA Mailing Address:

DARPA/DSO

ATTN: HR001119S0038

675 North Randolph Street Arlington, VA 22203-2114 o DARPA/DSO Opportunities Website: http://www.darpa.mil/work-with-us/opportunities

Frequently Asked Questions (FAQ): FAQs for this solicitation may be viewed on the DARPA/DSO Opportunities Website. See Section VIII.A for further information.

http://www.darpa.mil/work-with-us/opportunities

HR001119S0038 SAIL-ON 4

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

This Broad Agency Announcement (BAA) constitutes a public notice of a competitive funding opportunity as described in Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016. Any resultant negotiations and/or awards will follow all laws and regulations applicable to the specific award instrument(s) available under this BAA (e.g., FAR 15.4 for procurement contracts or 2 CFR § 200.203 for cooperative agreements). Proposals received as a result of this BAA will be evaluated in accordance with evaluation criteria specified herein through a scientific review process.

A. Introduction The Defense Sciences Office (DSO) of the Defense Advanced Research Projects Agency (DARPA) is soliciting innovative research proposals for new artificial intelligence (AI) methodologies and techniques that support: (1) the principled characterization and generation of novelty in open worlds; and (2) the creation of AI systems capable of operating appropriately and effectively in open worlds. Proposed research should investigate innovative approaches that enable revolutionary advances in science, devices, or systems. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice.

B. Background Military uses of AI are anticipated to be widespread. Military operations are typically characterized by novel situations, which arise in open worlds. We use the term “novelty” here to refer to situations that violate implicit or explicit assumptions about agents, the environment, or their interactions. As AI increasingly becomes ubiquitous for various aspects of military operations (including in decision support, human-machine collaboration, and autonomy), it will be essential for military AI applications to be aware of novelty in open worlds and capable of acting appropriately and effectively when confronted by novel situations.

Past AI approaches have not succeeded at handling novelty. Expert systems failed partially due to brittleness and the expense of recurrent updating needed as the world changed. Case-based reasoning was formulated to adapt past solutions to current situations, but in practice became retrieval of similar past cases without adaptation. Transfer learning requires careful engineering of the AI; it does not occur during the use of a system but rather before a system is deployed.

Deep learning is highly sensitive to imperceptibly changed examples. Well publicized successes such as AlphaGo Zero and Libratus generate many instances of training data with no changes to input or output features or interaction rules. Autonomous vehicles handle novel situations by retraining based on many miles of driving that include occurrence of previously unanticipated events.

Current research efforts in AI are characterized by: (1) a dependence on massive amounts of training data with no changes to input or output features, to interaction rules, and most important, to environment, goals, or context; (2) evaluation of ad-hoc approaches in which the introduced novelty is created by the researchers developing the solutions; and/or (3) no systematic qualitative and quantitative framework or measures of novelty.

Open set classification techniques have recently received attention from a handful of researchers.

HR001119S0038 SAIL-ON 5

While these techniques recognize the possibility of novel classes and can even partially identify such classes with reasonable accuracy, they are currently limited to well-defined tasks such as recognizing when letters are added to a dataset of images of digits or identifying species of animals that were not seen in the original training set; they do not apply to changes in environment, task, context, or goals that may occur during execution.

C. Program Description/Scope The Science of Artificial Intelligence and Learning for Open-world Novelty (SAIL-ON) program will research and develop the underlying scientific principles and general engineering techniques and algorithms needed to create AI systems that act appropriately and effectively in novel situations that occur in open worlds, which is a key characteristic needed for potential military applications of AI. The focus is on novelty that arises from violations of implicit or explicit assumptions in an agent’s model of the external world, including other agents, the environment, and their interactions. Specifically, the program will: (1) develop scientific principles to quantify and characterize novelty in open world domains; (2) create AI systems that act appropriately and effectively in open world domains; and (3) demonstrate and evaluate these systems in multiple domains, including a selected DoD domain.

To guide and evaluate the development of AI systems that can act appropriately and effectively in novel situations characteristic of open worlds, SAIL-ON is structured according to a domain-independent abstraction hierarchy that characterizes degrees and types of novelty that may be encountered by an intelligent agent in an open world. In the SAIL-ON program, novel situations are understood as those that violate implicit or explicit assumptions about the world where “the world” is understood to be a specified domain such as chess or a visual classifier for automobiles. An initial prototype of such a hierarchy is presented in Figure 1. It is recognized that this hierarchy may not be accurate in multiple aspects, such as the order of levels (indicated by the numbering in Figure 1), missing or redundant levels, the lack of inclusion of additional relevant dimensions (e.g., whether the novelty is local or global, the frequency with which novel situations occur, etc.), or others. This list is intended as a starting point, and proposers are encouraged to suggest improvements and corrections in their proposals and to use their modified version as a basis for their technical approach. To demonstrate the generality across domains of a proposer’s alternative hierarchy, examples should be provided for all levels in not only the proposer’s suggested domain(s) but also in at least one other domain. Alternative hierarchies may include additional or modified levels, different ordering, other factors or dimensions that are not addressed in the prototype, alternative definitions, or any other improvements that make the hierarchy more accurate or useful for understanding and advancing the research and development of open world AI capabilities. DARPA anticipates that further improvements beyond the version in Figure 1 and any proposed alternatives will result from the understanding gained during the SAIL-ON program research; an explicit goal of the program is to produce an accurate and accepted theory of novelty with both qualitative descriptions and quantitative measures corresponding to the difficulty imposed on AI systems.

HR001119S0038 SAIL-ON 6

Figure 1 Initial Prototype Open World Novelty Hierarchy

D. Program Structure SAIL-ON comprises three interacting Technical Areas (TAs) to be addressed in parallel: (TA1) Novelty Characterization and Generation; (TA2) Open World Capabilities; and (TA3) Evaluation and Transition. Proposals for TA1 and TA2 only are solicited by this BAA; TA3 will be conducted by a Government team comprised of FFRDC’s and/or Government Laboratories and is not being solicited by this BAA. A proposal may address either TA1 or TA2; a proposer wishing to respond to both TA1 and TA2 must submit separate proposals for each TA (see section III.D.1.).

The SAIL-ON program will be executed in three phases: 18 months for Phase 1 (base) to enable establishment of infrastructure and initial capabilities, and 12 months each for Phase 2 (option) and Phase 3 (option) to focus on capability improvements, increasingly difficult challenges, and evaluation of DoD relevance. In each phase of the program it is expected that the domain-independent characterization of novelty will be improved and that increasingly sophisticated and effective techniques for recognizing, characterizing, and responding to novel situations across domains will be developed.

There will be both mid-term evaluations and end-of-phase final evaluations. The mid-term evaluations will be used to understand the reasonableness and appropriateness of initial metrics and potentially to modify any overly ambitious values for the metrics. TA1 performers will facilitate the evaluations by providing novelty generators in the chosen domains, across levels of the novelty hierarchy. TA2 performers will need to show how their systems respond to the novelties introduced by TA1 generators. Initial performance metrics are described in Section E.

Proposals must address all three program phases, to include full technical and cost information.

Phases 2 & 3 should be proposed as fully priced options to be executed at the Government’s discretion based on funding availability and a performer’s progress against the SAIL-ON

HR001119S0038 SAIL-ON 7

program goals. All TA’s are planned for the full 42 month period of performance.

TA1 performers will be responsible for jointly (with all other TA1 performers) maintaining and updating the novelty hierarchy and associated metrics to characterize and quantify novelty in open world domains, including developing methods for quantifying novelty, and for constructing software that generates novel situations exemplifying multiple levels of the novelty hierarchy in chosen domains. TA2 performers will research and develop AI capabilities and systems that can identify and respond to novelty across the novelty hierarchy in multiple open world domains.

TA2 systems will be evaluated using TA1 novelty generating software. DARPA intends to select several distinct and diverse domains for use in the SAIL-ON program evaluations but only one TA1 performer per domain. Each TA1 domain will support at least two TA2 performer evaluations. Each TA2 performer’s capabilities and systems will be evaluated in multiple, but not necessarily all, domains.

The TA3 Government team will provide independent validation and verification of the TA2 systems in all phases of the program and will apply and evaluate TA2 technology to a capstone DoD application. In Phase 1, the TA3 Government team will develop and validate selection criteria for an appropriate DoD application for the SAIL-ON capstone evaluation and will identify and tradeoff potential DoD applications against these criteria. Based on this analysis, DARPA will select the capstone application. In Phase 2 of the program, the TA3 Government team will construct a novelty generator for the selected DoD capstone application, and in Phase 3 this team will enhance this novelty generator and conduct the DoD capstone evaluation and demonstration. The TA3 Government team will be responsible for any needed tailoring of the TA2 systems to support this evaluation. TA1 and TA2 teams are expected to provide technology and support to the TA3 Government team as needed.

E. Technical Area Descriptions Each Technical Area solicited by this BAA is described in detail below.

Technical Area 1 (TA1) Novelty Characterization and Generation:

TA1 performers will be responsible for: (1) jointly maintaining, evaluating, and updating a hierarchy and associated metrics to characterize and quantify novelty in multiple open world domains, including developing methods for quantifying novelty; and (2) for constructing a series of novelty generating software that generates novel situations exemplifying multiple levels of the novelty hierarchy in chosen domains. Each TA1 performer will develop a novelty generator for a single domain.

TA1 proposers must propose a domain for which an AI solution currently exists and explain their approach for generating and inserting a set of progressively more difficult novel situations corresponding to their proposed novelty hierarchy in this domain. For example, a proposer might propose the game of chess or Go, another board game, a specific card game or video game, object recognition or classification, activity labeling, or autonomous vehicle driving, or some other domain, and identify a corresponding existing AI capability, e.g., a chess or Go playing program, a poker playing program, a video game playing program, an object recognition program, or an autonomous driving program, respectively, for which baseline performance measures exist and which can serve as a baseline measure for performance in an unmodified

HR001119S0038 SAIL-ON 8

domain. The proposer must present a technical approach to creating novelty generators around this existing capability at all levels of the novelty hierarchy, to include examples of the types of novel situations that might be generated, such as changing the size of a game board, how pieces are allowed to move or interact, or objectives of a game.

Domains will vary in their characteristics. One could imagine domains which are deterministic or stochastic, single or multi-agent, have complete or incomplete knowledge of the environment and of opponent’s resources and state, and so forth. While preferred domain characteristics are not provided in the BAA, it is suggested that proposers include a clear statement of domain characteristics and provide technical details of how those characteristics will be leveraged when generating and inserting novel events into the domain. Note that the domains mentioned in this section or elsewhere in the BAA are for example purposes only; there is no a priori preference for these domains or for any other domain(s) that an proposer may propose other than their ability to support and enable research advances in open world novelty capabilities across the full set of novelty levels by multiple TA2 performers with distinct technical approaches. Generative approaches that are unpredictable are preferred over brute force approaches that select from a large set of pre-specified situations. The proposer’s approach must include a plan to specify an application programming interface (API) that will enable use by TA2 performers to evaluate their open world capability systems. Proposals must explain how their proposed domain will provide these abilities.

Implementation and use of novelty generators will provide the experimental evidence and experience for validating and improving the novelty hierarchy. To enable use of the novelty hierarchy by the DoD to design and evaluate open world AI systems, TA1 performers must also propose, apply, and validate qualitative and quantitative measures of novelty and a methodology for understanding the difficulty for AI applications in handling different levels of novelty. TA1 performers will need to be able to identify which level(s) of the novelty hierarchy that a specific generated novelty instantiates. TA1 performers will also need to be able to demonstrate that novelty generators are capable of generating novelties across their entire novelty hierarchy in quantifiable ways.

TA1 performers will be expected to collaboratively design and specify API’s for their novelty generators that enable their use by multiple TA2 performers. TA1 performers are also expected to collaborate and jointly maintain the novelty hierarchy used in the SAIL-ON program.

Proposals must address how the proposer will work with other SAIL-ON performers.

Key challenges for TA1 performers include validating the novelty hierarchy through experimentation, quantifying novelty at all levels of the hierarchy, exposing and making explicit inductive bias and other implicit assumptions of AI solutions in existing domains, defining API’s for novelty generators, and most importantly, ensuring that examples of generated novelty are not easily predictable by TA2 open world AI systems or researchers. Proposals need to explain how their approach will address these and any other identified challenges and provide quantified metrics for evaluating the quality of their novelty generators. Proposals must clearly identify technical risks and mitigation plans.

Technical Area 2 (TA2) Open World Capabilities:

HR001119S0038 SAIL-ON 9

The objective of TA2 is to develop AI systems with the capabilities needed to identify, characterize, and act appropriately and effectively to novel situations in open world domains.

TA2 proposals must provide a detailed phased technical approach for research and development leading to these open world AI capabilities. The approach must address identification, characterization, and action (or decision, if a static domain) generation capabilities at all levels of a novelty hierarchy. Just as in TA1, a proposal may suggest changes to the hierarchy that a proposer believes matches its phased approach to capabilities development. Capabilities need not be developed in the order corresponding to the initial hierarchy in Figure 1 if a proposer believes that an alternative order is more appropriate. Proposals must identify what information their systems might require from a novelty generator in order to generate appropriate actions. For example, a system might have to be told the level of a change that might occur, that a change has occurred, feedback that a hypothesized change has occurred, or other information that is required by a specific technical approach. Proposals must identify the feedback needed about the consequences of an action, if any, from a novelty generator or a human, when this feedback would be required, and what this means for the TA1/TA2 API. (An example of feedback might be the result of playing a hand in a card game.) Proposals must identify the domain(s) in which they intend to conduct their research and the domain characteristics to which their developed capabilities would apply. Proposers should describe how their approach minimizes or eliminates the need for explicit information from the novelty generator and the need for human intervention, as well as how it relies on data for training.

For purposes of evaluation, novelty situation instances are created by a TA1 developed novelty generator. Because a goal of the SAIL-ON program is general (i.e., domain-independent) open world AI techniques and systems, each TA2 developed open world AI system must be evaluated in multiple – but not all – program domains. DARPA recognizes that TA2 teams will need their own novelty generators to support their research, and encourages sharing of such novelty generators among TA2 performers. Proposals should explain how they would support such sharing.

Evaluation of open world AI capabilities will use separate metrics for identification and response to novelty. The number of examples of the novel situation (e.g., the number of moves made in chess after an objective changed, or the number of hands played in poker after sevens suddenly became wild) will be the independent variable. Dependent variables will include correct identification that a change occurred and effectiveness of the actions taken in response to the change. Proposals are encouraged to suggest additional measurements and metrics. Proposals must specify target and stretch goals for these metrics and provide a basis for these goals. An initial set of metrics and goals is presented in Table 1. More ambitious but realistic goals may be proposed, provided there is a credible approach to achieving them.

HR001119S0038 SAIL-ON 10

Table 1: Example TA2 performer metrics

Proposals must clearly identify technical risks and mitigations. Proposals must identify the characteristics and limitations of domains to which their approach may apply. Proposals must specify multiple exemplar domains that might be suitable for evaluation and application of their approach.

Potential approaches for TA2 might include various methods of machine learning, plan recognition, knowledge representation, anomaly detection, fault diagnosis and recovery, probabilistic programming, causal modeling, problem reformulation, theory/belief revision, goal reasoning, and others. There is no preference for approaches in this list compared to any other approach that may achieve the TA2 objectives.

F. Schedule/Milestones The overall program schedule and milestones are depicted in Figure 2.

Proposers should provide a technical and programmatic strategy that conforms to the provided program schedule and presents an aggressive plan to fully address all program goals, metrics, milestones and deliverables. The task structure must be consistent across the proposed schedule, Statement of Work, and cost volume. A target start date of October 1, 2019 may be assumed for planning purposes.

Schedules will be synchronized across performers, as required, and monitored/revised as necessary throughout the program.

All proposals must include the following meetings and travel in the proposed schedule and costs:

To continue integration and development between TAs, foster collaboration between teams, and disseminate program developments, a two to three day Principal Investigator (PI) meeting will be held approximately every six months, with locations split between the East and West Coasts of the United States. For budgeting purposes, plan for a 2 ½ day kickoff meeting in the Washington, DC area, and additional 2 ½ day meetings every 6 months alternating between the Washington, DC area and the San Francisco, CA area, HR001119S0038 SAIL-ON 11 with four meetings during phase 1 and 2 each during phases 2 and 3.

Regular teleconference meetings will be scheduled with the Government team for progress reporting as well as problems identification and mitigation. Proposers should also anticipate at least one site visit per year by the DARPA Program Manager during which they will have the opportunity to demonstrate progress towards agreed-upon milestones. For budgeting purposes, assume two site visits during Phase 1 of the program and one site visit during each of Phases 2 and 3.

Figure 2: Program schedule

G. Deliverables Performers will be expected to provide at a minimum the following deliverables:

Comprehensive quarterly technical reports due within ten days of the end of the given quarter, describing progress made on the specific milestones as laid out in the SOW.

A phase completion report submitted within 30 days of the end of each phase, summarizing the research done.

Other negotiated deliverables specific to the objectives of the individual efforts. These may include registered reports, experimental protocols, publications, data management plan, intermediate and final versions of software libraries, code, and APIs, including documentation and user manuals, and/or a comprehensive assemblage of design documents, models, modeling data and results, and model validation data.

Reporting as outlined in Section VI.C.

In addition to the above, TA1 performers must deliver:

HR001119S0038 SAIL-ON 12

A working evaluation domain with an accompanying novelty generator at the appropriate novelty levels instantiated as shown on the program schedule.

A revised novelty hierarchy that reflects current research every six months.

An assessment of the novelty generator’s ability to sample novelty at each level and across levels every six months.

In addition to the minimum deliverables above, TA2 performers must deliver:

A working system capable of being evaluated in the stated number of domains throughout the duration of the program.

Evaluation result reports due within two weeks of the completion of each evaluation.

H. Government-furnished Property/Equipment/Information None.

I. Other Program Objectives and Considerations

1. Collaboration

Throughout the course of the program, it will be necessary for all performers—regardless of category—to share relevant information regarding their research and development to support the larger program goals. DARPA expects all program performers to work collaboratively with one another to realize the program objectives outlined herein, so proposers should carefully review the goals for the entire program in order to fully understand the context of each program objective, performer category, and TA within the overall program structure. All proposals should describe plans for ensuring transparency of their processes to enable interactions with other program performers in Volume 1. Proposals that fail to include these plans may be deemed non-conforming and removed from consideration.

In particular, TA1 performers will need to provide their novelty generator software, documentation, and technical support to TA2 performers for use in evaluation, and TA2 performers will need to share the results of these evaluations, to include not only performance metrics and measurements but also feedback on the novelty generator itself to TA1 performers.

All TA1 performers and TA2 performers will jointly update the novelty hierarchy based on these results. TA1 and TA2 performers will need to provide code, documentation, and support to the TA3 Government team. TA1 and TA2 performers will need to agree on an API between TA1’s novelty generator and TA2’s open world AI system.

To ensure collaboration, all performer contracts will include an Associate Contractor Agreement (ACA) clause for portions of the contracts requiring joint participation between the TAs in the accomplishment of the program goals. This provision will become a material requirement for any agreements or contracts awarded as a result of this BAA. The ACA clause will include the basis for sharing information, data, technical knowledge, expertise and/or resources essential to the integration of the program technical areas and components. This clause will ensure appropriate coordination and integration of work by program contractors, ensure complete compatibility between data, tools and services, and prevent unnecessary duplication of efforts and maximize

HR001119S0038 SAIL-ON 13

commonality. Without exception, all ACAs must be in place within three months of contract award. See Section VIII.C for a sample ACA clause.

2. Intellectual Property Because SAIL-ON is a 6.1 basic research program, it is desirable that program products and knowledge be available for use by the broad AI research community. Therefore, intellectual property rights asserted by proposers are strongly encouraged to be aligned with open source regimes. See Section VI.B.4 for more information related to intellectual property. Any commercial software that is a necessary component of a performer’s approach must be identified and include a description of the license rights to be provided to the Government and how the license rights provided will enable use in any follow-on Government research programs.

II. Award Information A. General Award Information

DARPA anticipates multiple awards in each of TA1 and TA2, with fewer awards in TA1 than

TA2.

The level of funding for individual awards made under this BAA will depend on the quality of the proposals received and the availability of funds. Awards will be made to proposers1 whose proposals are determined to be the most advantageous to the Government, all evaluation factors considered. See Section V for further information.

The Government reserves the right to:

select for negotiation all, some, one, or none of the proposals received in response to this solicitation;

make awards without discussions with proposers;

conduct discussions with proposers if it is later determined to be necessary;

segregate portions of resulting awards into pre-priced options;

accept proposals in their entirety or to select only portions of proposals for award;

fund awards in increments with options for continued work at the end of one or more phases;

request additional documentation once the award instrument has been determined (e.g., representations and certifications); and remove proposers from award consideration should the parties fail to reach agreement on award terms within a reasonable time or the proposer fails to provide requested additional information in a timely manner.

1 As used throughout this BAA, “proposer” refers to the lead organization on a submission to this BAA. The proposer is responsible for ensuring that all information required by a BAA--from all team members--is submitted in accordance with the BAA. “Awardee” refers to anyone who might receive a prime award from the Government, including recipients of procurement contracts, cooperative agreements, or Other Transactions. “Subawardee” refers to anyone who might receive a subaward from a prime awardee (e.g., subawardee, consultant, etc.).

HR001119S0038 SAIL-ON 14

Proposals identified for negotiation may result in a procurement contract, cooperative agreement, or other transaction (OT), depending upon the nature of the work proposed, the required degree of interaction between parties, and other factors.

Proposers looking for innovative, commercial-like contractual arrangements are encouraged to consider requesting Other Transactions. To understand the flexibility and options associated with Other Transactions, consult http://www.darpa.mil/work-with-us/contract-management#OtherTransactions.

In accordance with 10 U.S.C. § 2371b(f), the Government may award a follow-on production contract or Other Transaction (OT) for any OT awarded under this BAA if: (1) that participant in the OT, or a recognized successor in interest to the OT, successfully completed the entire prototype project provided for in the OT, as modified; and (2) the OT provides for the award of a follow-on production contract or OT to the participant, or a recognized successor in interest to the OT.

In all cases, the Government contracting officer shall have sole discretion to select award instrument type, regardless of instrument type proposed, and to negotiate all instrument terms and conditions with selectees. DARPA will apply publication or other restrictions, as necessary, if it determines that the research resulting from the proposed effort will present a high likelihood of disclosing performance characteristics of military systems or manufacturing technologies that are unique and critical to defense. Any award resulting from such a determination will include a requirement for DARPA permission before publishing any information or results on the program. For more information on publication restrictions, see the section below on Fundamental Research.

B. Fundamental Research

It is DoD policy that the publication of products of fundamental research will remain unrestricted to the maximum extent possible. National Security Decision Directive (NSDD) 189 defines fundamental research as follows:

‘Fundamental research’ means basic and applied research in science and engineering, the results of which ordinarily are published and shared broadly within the scientific community, as distinguished from proprietary research and from industrial development, design, production, and product utilization, the results of which ordinarily are restricted for proprietary or national security reasons.

As of the date of publication of this BAA, the Government expects that program goals as described herein may be met by proposers intending to perform fundamental research and does not anticipate applying publication restrictions of any kind to individual awards for fundamental research that may result from this BAA. Notwithstanding this statement of expectation, the Government is not prohibited from considering and selecting research proposals that, while perhaps not qualifying as fundamental research under the foregoing definition, still meet the BAA criteria for submissions. If proposals are selected for award that offer other than a fundamental research solution, the Government will either work with the proposer to modify the proposed statement of work to bring the research back into line with fundamental research or else the proposer will agree to restrictions in order to receive an award.

http://www.darpa.mil/work-with-us/contract-management#OtherTransactions http://www.darpa.mil/work-with-us/contract-management#OtherTransactions

HR001119S0038 SAIL-ON 15

Proposers should indicate in their proposal whether they believe the scope of the research included in their proposal is fundamental or not. While proposers should clearly explain the intended results of their research, the Government shall have sole discretion to determine whether the proposed research shall be considered fundamental and to select the award instrument type. Appropriate language will be included in resultant awards for non-fundamental research to prescribe publication requirements and other restrictions, as appropriate. This language can be found at http://www.darpa.mil/work-with-us/additional-baa.

For certain research projects, it may be possible that although the research to be performed by a potential awardee is non-fundamental research, its proposed subawardee’s effort may be fundamental research. It is also possible that the research performed by a potential awardee is fundamental research while its proposed subawardee’s effort may be non-fundamental research.

In all cases, it is the potential awardee’s responsibility to explain in its proposal which proposed efforts are fundamental research and why the proposed efforts should be considered fundamental research.

III. Eligibility Information A. Eligible Applicants

All responsible sources capable of satisfying the Government's needs may submit a proposal DARPA’s consideration.

1. Federally Funded Research and Development Centers (FFRDCs) and Government Entities

a. FFRDCs

FFRDCs are subject to applicable direct competition limitations and cannot propose to this BAA in any capacity unless they meet the following conditions: (1) FFRDCs must clearly demonstrate that the proposed work is not otherwise available from the private sector. (2) FFRDCs must provide a letter on official letterhead from their sponsoring organization citing the specific authority establishing their eligibility to propose to Government solicitations and compete with industry, and their compliance with the associated FFRDC sponsor agreement’s terms and conditions. This information is required for FFRDCs proposing to be awardees or subawardees.

b. Government Entities

Government Entities (e.g., Government/National laboratories, military educational institutions, etc.) are subject to applicable direct competition limitations. Government entities must clearly demonstrate that the work is not otherwise available from the private sector and provide written documentation citing the specific statutory authority and contractual authority, if relevant, establishing their ability to propose to Government solicitations. This information is required for Government Entities proposing to be awardees or subawardees.

http://www.darpa.mil/work-with-us/additional-baa

HR001119S0038 SAIL-ON 16

c. Authority and Eligibility

At the present time, DARPA does not consider 15 U.S.C. § 3710a to be sufficient legal authority to show eligibility. While 10 U.S.C.§ 2539b may be the appropriate statutory starting point for some entities, specific supporting regulatory guidance, together with evidence of agency approval, will still be required to fully establish eligibility. DARPA will consider FFRDC and Government entity eligibility submissions on a case-by-case basis; however, the burden to prove eligibility for all team members rests solely with the proposer.

2. Foreign Participation Non-U.S. organizations and/or individuals may participate to the extent that such participants comply with any necessary nondisclosure agreements, security regulations, export control laws, and other governing statutes applicable under the circumstances. For classified submissions, this includes mitigating any Foreign Ownership Control and Influence (FOCI) issues prior to transmitting the submission to DARPA. Additional information on these subjects can be found at http://www.dss.mil/isp/foci/foci_faqs.html.

B. Organizational Conflicts of Interest

FAR 9.5 Requirements In accordance with FAR 9.5, proposers are required to identify and disclose all facts relevant to potential OCIs involving the proposer’s organization and any proposed team member (subawardee, consultant). Under this Section, the proposer is responsible for providing this disclosure with each proposal submitted to the BAA. The disclosure must include the proposer’s, and as applicable, proposed team member’s OCI mitigation plan. The OCI mitigation plan must include a description of the actions the proposer has taken, or intends to take, to prevent the existence of conflicting roles that might bias the proposer’s judgment and to prevent the proposer from having unfair competitive advantage. The OCI mitigation plan will specifically discuss the disclosed OCI in the context of each of the OCI limitations outlined in FAR 9.505-1 through FAR 9.505-4.

Agency Supplemental OCI Policy In addition, DARPA has a supplemental OCI policy that prohibits contractors/performers from concurrently providing Scientific Engineering Technical Assistance (SETA), Advisory and Assistance Services (A&AS) or similar support services and being a technical performer.

Therefore, as part of the FAR 9.5 disclosure requirement above, a proposer must affirm whether the proposer or any proposed team member (subawardee, consultant) is providing SETA, A&AS, or similar support to any DARPA office(s) under: (a) a current award or subaward; or (b) a past award or subaward that ended within one calendar year prior to the proposal’s submission date.

If SETA, A&AS, or similar support is being or was provided to any DARPA office(s), the proposal must include:

The name of the DARPA office receiving the support;

HR001119S0038 SAIL-ON 17

The prime contract number;

Identification of proposed team member (subawardee, consultant) providing the support; and An OCI mitigation plan in accordance with FAR 9.5.

Government Procedures In accordance with FAR 9.503, 9.504 and 9.506, the Government will evaluate OCI mitigation plans to avoid, neutralize or mitigate potential OCI issues before award and to determine whether it is in the Government’s interest to grant a waiver. The Government will only evaluate OCI mitigation plans for proposals that are determined selectable under the BAA evaluation criteria and funding availability.

The Government may require proposers to provide additional information to assist the Government in evaluating the proposer’s OCI mitigation plan.

If the Government determines that a proposer failed to fully disclose an OCI; or failed to provide the affirmation of DARPA support as described above; or failed to reasonably provide additional information requested by the Government to assist in evaluating the proposer’s OCI mitigation plan, the Government may reject the proposal and withdraw it from consideration for award.

Include any OCIs affirmations and disclosures in Attachment G Proposal Template Vol. 3- Admin and National Policy Requirements.

C. Cost Sharing/Matching Cost sharing is not required; however, it will be carefully considered where there is an applicable statutory condition relating to the selected funding instrument (e.g., OTs under the authority of 10 U.S.C. § 2371). Cost sharing is encouraged where there is a reasonable probability of a potential commercial application related to the proposed research and development effort.

For more information on potential cost sharing requirements for Other Transactions for Prototype, see http://www.darpa.mil/work-with-us/contract-management#OtherTransactions.

D. Other Eligibility Requirements

1. Ability to Receive Awards in Multiple Technical Areas - Conflicts of Interest

Proposers may submit to either or both Technical Areas 1 and 2. Separate proposals are required for each TA. Any proposer who submits to both TA1 and TA2 must include in both proposals, potentially with different levels of detail as appropriate for the method that is proposed, an explanation of how no evaluation advantage will be gained by the proposer’s TA2 system against their TA1 novelty generator compared to any other TA2 performer’s system evaluated against the proposer’s TA1 novelty generator. The purpose of this is not only to provide for a fair and equitable evaluation, but also to ensure that the TA1 novelty generator is not biased in any unknown or subtle way towards any TA2 approach. Potential methods for avoiding advantaging a proposer’s TA2 evaluation might include a specific plan to avoid information flow from their TA2 team researchers to their TA1 researchers independent of any other TA2 team’s researchers, a commitment to use TA1 evaluation domains from other selected TA1 performers, or any other method that may be effective. There is no preference for these exemplar methods. Proposers who

HR001119S0038 SAIL-ON 18

elect to propose to both TA1 and TA2 must submit separate proposals for both efforts so DARPA can select their proposal for either or both TA’s.

2. Ability to Support Classified Development No classified work is anticipated under TA1 or TA2 of this BAA.

IV. Application and Submission Information Prior to submitting a full proposal, proposers are strongly encouraged to first submit an abstract as described below. This process allows a proposer to ascertain whether the proposed concept is: (1) applicable to the SAIL-ON BAA and (2) currently of interest. For the purposes of this BAA, applicability is defined as follows:

The proposed concept is applicable to the technical areas described herein.

The proposed concept is important to DSO’s current investment portfolio.

The proposed concept investigates an innovative approach that enables revolutionary advances, i.e., will not primarily result in evolutionary improvements to the existing state of practice.

The proposed work has not already been completed (i.e., the research element is complete but manufacturing/fabrication funds are required).

The proposer has not already received funding or a positive funding decision for the proposed concept (whether from DARPA or another Government agency).

Abstracts and full proposals that are not found to be applicable to the SAIL-ON BAA as defined above may be deemed non-conforming2 and removed from consideration. All abstracts and full proposals must provide sufficient information to assess the validity/feasibility of their claims as well as comply with the requirements outlined herein for submission formatting, content and transmission to DARPA. Abstracts and full proposals that fail to do so may be deemed non-conforming and removed from consideration. Proposers will be notified of non-conforming determinations via letter.

A. Address to Request Application Package This document contains all information required to submit a response to this solicitation. No additional forms, kits, or other materials are needed except as referenced herein. No request for proposal or additional solicitation regarding this opportunity will be issued, nor is additional information available except as provided at the Federal Business Opportunities website (http://www.fbo.gov), the Grants.gov website (http://www.grants.gov/), or referenced herein.

B. Content and Form of Application Submission

1. Abstract Information

2 “Conforming” is defined as having been submitted in accordance with the requirements outlined herein.

http://www.fbo.gov/ http://www.onr.navy.mil/Contracts-Grants/submit-proposal/grants-proposal/grants-terms-conditions.aspx

HR001119S0038 SAIL-ON 19

As stated above, proposers are strongly encouraged to submit an abstract in advance of a full proposal to minimize effort and reduce the potential expense of preparing an out of scope proposal. The abstract provides a synopsis of the proposed project by briefly answering the following questions:

What is the proposed work attempting to accomplish or do?

How is it done today, and what are the limitations?

Who will care and what will the impact be if the work is successful?

How much will it cost, and how long will it take?

DARPA will respond to abstracts with a statement as to whether DARPA is interested in the idea. If DARPA does not recommend the proposer submit a full proposal, DARPA will provide feedback to the proposer regarding the rationale for this decision. Regardless of DARPA’s response to an abstract, proposers may submit a full proposal. DARPA will review all conforming full proposals using the published evaluation criteria and without regard to any comments resulting from the review of an abstract.

Proposers should note that a favorable response to an abstract is not a guarantee that a proposal based on the abstract will ultimately be selected for award negotiation.

While it is DARPA policy to attempt to reply to abstracts within thirty calendar days, proposers to this solicitation may anticipate a response within approximately three weeks. These official notifications will be sent via email to the Technical POC and/or Administrative POC identified on the abstract coversheet.

Abstract Format All proposers are required to use Attachment A: Abstract Summary Slide Template and Attachment B: Abstract Template provided to this solicitation on http://www.fbo.gov and http://www.grants.gov. Attachment A Abstract Summary Slide Template described herein must be in .ppt or .pptx format and should be attached as a separate file to this document.

2. Full Proposal Information Proposals consist of Volume 1: Technical and Management Volume, Volume 2: Cost Volume, and Volume 3: Administrative and National Policy Requirements Volume.

To assist in proposal development, various attachments have been provided along with the BAA posted on http://www.fbo.gov (Attachment C: Proposal Summary Slide Template; Attachment D: Proposal Template Volume 1 Technical & Management Volume; Attachment E: Proposal Template Volume 2 Cost Volume; Attachment F: Proposal Template Volume 2 Cost Breakdown Template; Attachment F-2: Proposal Template Vol. 2-Cost Breakdown (optional); and Attachment G: Proposal Template Volume 3 Administrative & National Policy Requirements Volume).

https://wawf.eb.mil/ http://www.grants.gov/

HR001119S0038 SAIL-ON 20

Full proposals requesting a procurement contract or other transaction (OT) must use the following attachments:

Attachment C Attachment D Attachment E Attachment F Attachment F-2 Attachment G

Full proposals requesting a cooperative agreement must use the following attachments in addition to the Grants.gov application package:

Attachment C Attachment D Attachment F Attachment F-2 Attachment G *Note – Budget Justification should be provided as Section L of the SF 424 Research & Related Budget form provided via Grants.gov.