HR001119S0034.pdf

Broad Agency Announcement Artificial Social Intelligence for Successful Teams (ASIST)

HR001119S0034

March 18, 2019

Defense Advanced Research Projects Agency Information Innovation Office 675 North Randolph Street Arlington, VA 22203-2114

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 2

Table of Contents

I. Funding Opportunity Description

A. Introduction/Background

B. Program Description/Scope

C. Program Structure

D. Technical Areas

E. Schedule/Milestones

F. Deliverables

G. Government-furnished Property/Equipment/Information

H. Intellectual Property

I. Potential Conflicts of Interest

II. Award Information

A. Awards

B. Fundamental Research

C. Disclosure of Information and Compliance with Safeguarding Covered Defense Information Controls

III. Eligibility Information

A. Eligible Applicants

B. Organizational Conflicts of Interest

C. Cost Sharing/Matching

IV. Application and Submission Information

A. Address to Request Application Package

B. Content and Form of Application Submission

C. Submission Dates and Times

D. Funding Restrictions

E. Other Submission Requirements

V. Application Review Information

A. Evaluation Criteria

B. Review and Selection Process

VI. Award Administration Information

A. Selection Notices

B. Administrative and National Policy Requirements

C. Reporting

VII. Agency Contacts

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 3

VIII. Other Information

A. Frequently Asked Questions (FAQs)

B. Proposers Day

C. Submission Checklist

D. Associate Contractor Agreement (ACA)

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 4

PART I: OVERVIEW INFORMATION

Federal Agency Name: Defense Advanced Research Projects Agency (DARPA), Information Innovation Office (I2O)

Funding Opportunity Title: Artificial Social Intelligence for Successful Teams

(ASIST)

Announcement Type: Initial Announcement

Funding Opportunity Number: HR001119S0034

Catalog of Federal Domestic Assistance Numbers (CFDA): 12.910 Research and Technology Development

Dates o Posting Date: March 18, 2019 o Proposers Day: March 14, 2019 o Abstract Due Date: April 2, 2019, 12:00 noon (ET) o Proposal Due Date: May 17, 2019, 12:00 noon (ET) o BAA Closing Date: May 17, 2019, 12:00 noon (ET)

Anticipated Individual Awards: DARPA anticipates 4-6 awards for Technical Area 1 (TA1), many smaller awards for TA2, and a single award for TA3.

Types of Instruments that may be awarded: Procurement contracts, grants, Other Transactions (OTs), or cooperative agreements.

Agency Contacts o Technical POC: Joshua Elliott, Program Manager, DARPA/I2O o BAA Email: ASIST@darpa.mil o BAA Mailing Address:

DARPA/I2O

ATTN: HR001119S0034

675 North Randolph Street Arlington, VA 22203-2114 o I2O Solicitation Website: http://www.darpa.mil/work-with-us/opportunities http://www.darpa.mil/work-with-us/opportunities

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 5

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

DARPA is soliciting innovative research proposals in the area of machine social intelligence in a teaming context. Proposed research should investigate innovative approaches that enable revolutionary advances in science, devices, or systems. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice.

This Broad Agency Announcement (BAA) is being issued, and any resultant selection will be made, using procedures under Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016.

Any negotiations and/or awards will use procedures under FAR 15.4 (or 32 CFR § 200.203 for grants and cooperative agreements). Proposals received as a result of this BAA shall be evaluated in accordance with evaluation criteria specified herein through a scientific review process.

DARPA BAAs are posted on the Federal Business Opportunities (FBO) website (https://www.fbo.gov/) and the Grants.gov website (https://www.grants.gov/).

The following information is for those wishing to respond to this BAA.

A. Introduction/Background

Artificial Intelligence (AI) technologies have made little progress in understanding the most important component of the environments in which they operate: humans. This lack of understanding stymies efforts to create safe, efficient, and productive human-machine teams.

The Artificial Social Intelligence for Successful Teams (ASIST) program will develop foundational AI theory and systems that demonstrate the basic machine social skills needed to infer the goals and situational knowledge (i.e., what they believe, correctly or not, to be true of their current situation) of human partners, predict what they will need, and offer context aware actions in order to perform as adaptable and resilient AI teammates.

What makes humans good teammates? Humans intuitively combine pre-existing knowledge with observations and contextual clues to construct rich mental models of the world around them and use these models to evaluate goals, perform thought experiments, make predictions, and update their situational understanding. When the environment contains other people, humans use a skill called theory of mind (ToM) to infer their mental states from observed actions and context, and predict future actions from those inferred states. When humans form teams, these models can become extremely complex. High-performing teams naturally align key aspects of their models to create shared mental models of their environment, equipment, team, and strategies. ToM and the ability to create shared mental models are key elements of human social intelligence. Together, these two skills form the basis for human collaboration at all scales, whether the setting is a playing field or a military mission.

ASIST performers will create agents that demonstrate these social skills: a machine ToM and the ability to participate in an effective team by representing and helping to maintain shared models.

The program will create and employ a testbed for evaluating these agents using customizable open-world environments and standardized interfaces. The interfaces will include a package of https://www.fbo.gov/

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 6

standard sensing channels (i.e., information streams from physical and virtual sensors that will be available to ASIST agents) and communication/action channels (i.e., mechanisms for agents to convey information to human teammates in the testbed and what options they can use to engage with the team). ASIST agents must operate in increasingly complex and specialized environments, be adaptable to sudden perturbations in the mission or team (like the loss of communication with a key teammate), and use noisy multi-channel observations to represent the world and do complex inference and prediction.

B. Program Description/Scope

DARPA is seeking revolutionary ideas that lead to rigorous and practical demonstrations of the feasibility and effectiveness of advances in machine social intelligence in a teaming context.

ASIST envisions agents that observe their surroundings; build and maintain rich representations of the environment, team, and individuals, infer their teammates’ situational knowledge and goals; predict their teammates’ actions and needs; and assist the team by planning complex actions and executing them at appropriate times. ASIST agents will operate in increasingly complex and specialized environments, and will adapt to unexpected perturbations such as a sudden disruption to the team or change in strategy.

In addition, ASIST will advance the scientific state-of-the-art in the area of human-machine teaming. There are many unanswered questions about human-AI teaming, and addressing them will require multiple perspectives from fields including, but not limited, to cognitive science, team science, human factors, and organizational studies.

ASIST agents will be demonstrated in a virtual testbed in which both humans and agents can interact. Experiments will be designed in this testbed to evaluate agents and test novel hypotheses about human and human-machine teaming.

Proposals describing approaches for agents performing services in the physical world are not of interest and are considered out of scope for the program. Also not of interest for ASIST are proposals that focus on development of sensors to enable agents to observe their surroundings.

C. Program Structure

The ASIST program will consist of three technical areas (TAs):

TA1: AI Agent Architectures TA2: Cognitive Modeling of Individuals and Teams TA3: Environment and Evaluation

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 7

Figure 1: Structure of ASIST showing the interaction of different technical areas

Figure 1 depicts the structure of ASIST, including the various reasoning processes that must be addressed. Each proposal may only address a single technical area but proposers may submit multiple proposals. A proposer may submit proposals for all three technical areas and may participate in either or both TA1 and TA2; however, a proposer selected for TA3 will be unable to perform on any portion of TA1 and/or TA2. This selection process is intended to avoid organizational conflicts of interest (OCI) situations between TAs 1 and 2 and the integration and evaluation activities in TA3, as well as to ensure objective test and evaluation results. The decision as to which proposal to select for award is at the discretion of the Government.

TA3 is expected to involve Human Subjects Research (HSR). TA1 and TA2 may include HSR.

See Sections IV.B.2 and VI.B.2 for further HSR proposal submission details.

ASIST will have three phases. The first and second phases will be 15 months each, and the third phase will be 18 months long for a total program length of 48 months, or four years. Phase 1 will focus on technologies to create agents that interact with a single human partner. Phase 2 will extend Phase 1 technologies to include interactions with multiple humans on a team. Phase 3 will research and develop agents capable of partnering with complex teams comprising individuals with specialized roles and skills.

Each phase will have two technical evaluation exercises for a total of six evaluation events throughout the duration of the program. The first evaluation of each phase will occur in the 9th month of the phase and involve relatively simple tasks and only one class of perturbation (e.g., HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 8 reward revaluations only). The second evaluation of the phase will occur at phase completion, and involve relatively complex tasks and multiple classes of perturbation. TA3 will develop and execute the evaluation exercises. TA3 will also conduct evaluation dry runs with the TA1 and TA2 performers approximately three months before each evaluation, to ensure the government team has sufficient insight into team progress to make adjustments to the experiments as needed.

No forced down-selects are anticipated. However, to continue from one phase to the next, individual performers need to demonstrate the viability of their technical approaches and progress toward producing technologies that support the goals and objectives of the program.

All performers will be required to agree to an Associate Contractor Agreement (ACA), in order to facilitate program-wide collaboration. TA3 will lead the process for establishing the ACA.

D. Technical Areas

TA1: AI Agent Architectures

The goal of TA1 is to develop and implement AI architectures for human-machine teaming. The agent architectures must:

use generalizable approaches to represent teams and individual partners;

demonstrate real-time scalable inference and predict the actions of human partners;

demonstrate the ability to scale to increasingly complex operations; and handle multiple classes of perturbations effectively.

By the end of Phase 1, TA1 teams must demonstrate effective AI agent architectures that exhibit key aspects of machine social intelligence. More specifically, using multi-channel observations, agents should develop representations of the environment and of their human teammate that are sufficient to infer unobservable states of human teammates (such as their situational knowledge, policies, and goals), predict their subsequent actions and needs, and autonomously deliver appropriate information and services to their human teammates. These agents should also have sufficient self-awareness to model how their actions change the environment and team system.

By the end of Phase 2, the agents must demonstrate machine social intelligence that includes interactions between multiple humans (for example, the ability to participate in the team’s shared mental models). Finally in Phase 3, the agent must make useful contributions to a team of humans, each with specialized roles and skills working collectively toward a complex objective.

ASIST agents must be able to accept and use structured mission knowledge from the team (e.g., rules, constraints, norms, and strategies) and use observations and analytics to reason over team and teammate goals, states, and actions. It is expected that ASIST agents will learn over time to recognize and encourage desirable properties of teams, such as effective team dynamics, efficient task loads, and improved strategies. Agents should also create individualized profiles of human teammates that include categories such as expertise, limitations, effective roles, and common behaviors; this knowledge should be employed to improve their models of the humans/team and the effectiveness of services provided. Strong proposals will describe architectures that can operate in both local and distributed teaming environments, where available sensing and communication options will be limited.

TA1 performers will participate in a working group convened by TA3 (see section regarding TA3) to design the challenge environment and standardized interface, including available

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 9

sensing and communication channels. TA1 teams may propose specific API elements to extract state information from the challenge environment, software sensors to monitor operator input devices, and non-invasive sensors to monitor the physiological status of operators. For sensors that are difficult to use or not commonly available, TA1 performers will be expected to provide the sensors. TA1 proposals should address how agents will adapt to environments where some sensors are unavailable (e.g., distributed virtual teams). ASIST is not interested in proposals that include significant research effort for novel sensors.

The ASIST working group will also design a standard set of communication mechanisms with which agents can effectively communicate information to their human teammates and participate in alignment of the shared models (e.g. text, audio, visuals, dashboards, etc.). This will include available action options such as specific types and frequencies of knowledge that can be provided.

TA2: Cognitive Modeling and Individuals and Teams

The primary objective for TA2 teams is to formulate novel and testable hypotheses about human-machine teaming and human social cognition. This is expected to include critical topics, such as the variability in team performance (e.g., based on measures of latent properties of teams), norms of behavior and communication, trust, nested mental models, and the applicability to hybrid teams of successful theoretical frameworks for human team performance. TA2 will develop theories of effective human-machine teaming, and will work closely with TA1 to inform the design of the agent architectures. TA2 proposals should describe the state-of-the-art and how it will be advanced.

TA2 performers will pre-register testable hypotheses (to include the approximate magnitude and direction of the effect size) in a repository managed by TA3. TA2 proposers should describe how they will minimize the ambiguity of their claims, and how they will be measured. Proposals should also describe methods for verifying the robustness of analyses.

TA2 teams will collaborate with TA3 to develop new measures and experiments to test these hypotheses, and will analyze and publish the resulting data and conclusions. In so doing, TA2 will contribute to every aspect of the program, enabling improvements in TA1 architectures, and significantly advancing the state of our understanding of human-human and human-machine teaming.

TA2 proposals should include a detailed description of theoretical perspectives and structures, along with existing published claims from the literature that can be tested in the TA3 testbed.

Strong proposals will present initial claims that already have evidence from the literature, so that the verification of these claims in the ASIST testbed can provide a firm theoretical grounding for the program. TA2 may propose exploratory experimentation if needed, but TA3 will conduct all confirmatory experimentation (see the section regarding TA3). After each evaluation, TA2 will analyze the resulting data, publish their analyses and conclusions, and propose new and/or revised hypotheses to test in the next evaluation cycle.

TA2 performers will be evaluated based on theories that were validated or invalidated, and published scholarship. TA2 performers are strongly encouraged to work closely with TA1 and TA3 teams to understand what will be experimentally available and to inform the design of agent architectures and experiments.

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 10

TA3: Environment and Evaluation

The TA3 performer will develop and implement a scalable testbed and challenge environment(s), organize and execute tests in this testbed, and coordinate with performers to assure protocols and APIs are stable and functioning prior to evaluations and dry-runs. The testbed must include environments that allow for both simple and complex challenges with a single human (Phase 1), many humans (teams as large as 10 in Phase 2), and complex teams with individuals possessing specialized roles and skills (Phase 3).

The TA3 performer will organize, design, and execute six evaluations (two per phase), and host technical meetings approximately three months before each evaluation. The two-day evaluation events will be hosted at the site of the testbed (though they may include both local and distributed experiments) and include representatives from TA1 and TA2 teams. Technical meetings will include a three-day dry-run experiment and a two-day all-hands principal investigator (PI) meeting (plan for attendance of roughly 75-100 people). Location for technical meetings will be local to the TA3 performer. The primary objective of the dry-run is to finalize systems engineering, protocols, and standards in preparation for the subsequent evaluation. It is expected that a small amount of data will be collected at the dry-run events that TA2 teams can use to design analyses (thus proposals should address the likelihood that dry-runs will entail HSR). In addition to the final evaluation event at month 47, TA3 will also organize and host a final three-day technology demonstration in conjunction to a two-day final PI meeting at month 48 in the Washington D.C. area (see Schedule and Milestones below for more details). TA3 proposers should discuss how they will facilitate these events, including the acquisition and provisioning of appropriate event facilities and resources.

TA3 will convene a working group at the beginning of each program phase, with representation from each TA1 and TA2 performer team, to establish a framework for experiments, requirements for a common interface to the testbed environment, and priorities to accommodate as many of the requirements of the other performer teams as feasible. TA3 proposals should include a baseline cost to provide software integration support to four TA1 performer teams as well as experimental planning support to four TA2 performer teams. Proposal should also include separate optional tasks to support additional performer teams in TA1 or TA2. The interface will include a package of standard sensing channels (i.e., information streams that will be available to TA1 agent architectures) and communication/action channels (i.e., mechanisms for agents to convey information to human teammates in the testbed and what options they can use to engage with the team). The sensing channels may include streams from specialized sensors requested and provided by TA1 and TA2 performer teams, but will always include a core set of sensors that are practical in a distributed team environment. The experiment framework agreed to during these working groups is expected to include the structure of the environment and tasks, and a set of additional measures implemented in the experiments to facilitate tests of TA2 hypotheses (which are likely to require measurements outside the recorded sensing channels such as survey instruments or other tests).

TA3 will produce passive experimental designs, in which humans and agents monitor human teams (or recordings of the teams) in the challenge environment, and make inferences and predictions but do not intervene, and active experimental designs in which agents contribute to the team.

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 11

Proposed testbeds must be able to challenge human and hybrid teams with both simple and complex tasks. An example of such a testbed would be a customizable Minecraft1 environment (though this example is not intended to be prescriptive, and proposals should identify the environment or environments that will best meet the program’s objectives). Proposed testbeds must have the ability to implement multiple classes of perturbations. Perturbations might include, but are not limited to Reward Revaluation (e.g., sudden changes in mission goal), Transition Revaluation (e.g., structural changes to the environment or team), and Policy Revaluation (e.g., a sudden change to the team strategy).

Strong TA3 proposals will adapt and extend an existing AI testbed with environments that are accessible by distributed teams and can be sustainably maintained as an open community resource for continued advancement in human-machine teaming after the program completes.

After each evaluation, the challenge environments should be made available to performers for continued testing. Strong proposals will discuss ways in which to make the testbed environments available to TA1 performers early in Phase 1.

TA3 will maintain a repository in which TA2 performer teams will pre-register testable hypotheses, context, and experimental results and analyses. Strong TA3 proposals will discuss approaches for instantiating this repository to make it available early in the program. TA3 will conduct all experiments, ensure the rigor of the experimental environment, and will collaborate with TA2 performers to ensure academic rigor in the experiment design. In so doing, TA3 should facilitate transfer of best experimentation practices across the TA2 teams and refine their experimentation environment to support a wide range of experiments. By the end of the program, this experimentation environment and its datasets should be sufficiently mature that it can be made open source as a resource to the theoretical and experimental community.

The evaluation exercises developed and conducted by TA3 must assess the Agent’s social skill and the Team’s effectiveness. Example measures to quantify during evaluation exercise include but are not limited to accuracy of inferences and predictions, time to generate initial inference or update with new information, agent usefulness, trust, adaptation and resilience time, and coordination effort required after perturbation. Strong proposals will discuss additional relevant metrics and measures to be quantified during evaluation exercises.

The TA3 performer will be responsible for recruiting all human subjects that will participate in the evaluation exercises. Sample sizes should be large enough to produce statistically meaningful results for the metrics of Agent Social Skill and Team Effectiveness. The TA3 performer will also be responsible for leading the establishment of the Associate Contractor Agreement.

E. Schedule/Milestones

The schedule for ASIST is shown in Figure 2. The program will start with a 3-day kickoff meeting hosted by DARPA. The program will end with a 5-day meeting that includes a final 3-day technology demonstration followed by a 2-day final PI meeting located in Washington D.C, and hosted by TA3.

1 Minecraft is a multiplayer, collaborative video game platform.

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 12

Six technical meetings will be conducted approximately three months before each evaluation event. Technical meetings will last five days, and include a 3-day dry-run evaluation and a 2-day all-hands PI meeting. Technical meetings will be conducted at a location chosen by the TA3 performer; therefore, travel by other TAs to the evaluation site will be required. For budgeting purposes, please assume either an east or west coast location, whichever is further from a proposer’s home location.

Six 2-day evaluation events are anticipated to occur on months 9 and 15 (Phase 1), 24 and 30 (Phase 2), and 39 and 47 (Phase 3) of the program. Evaluations will be conducted at a location chosen by the TA3 performer; therefore, travel by other TAs to the evaluation site will be required. For budgeting purposes, please assume either an east or west coast location, whichever is further from a proposer’s home location.

The program manager and other U.S. Government stakeholders will visit prime performer sites approximately once per phase. Visits are anticipated to last one day, and the prime performer should plan for key personnel to participate.

01-03 04-06 07-09 10-12 13-15 16-18 19-21 22-24 25-27 28-30 31-33 34-36 37-39 40-42 43-45 46-48

T ec hnic al A reas

H S

R

T A1: AI Ag ent Arc hitec tures

T A2: C og nitive Modeling of Individuals and T eams

E valuation

T A3: E nvironment and

E valuation

E valuations

Dry R uns

K ickoff and P I Meetings indicates final technology demonstration

18 Months

E xpand agent social skills to complex human teams in specialized environment

Analyze and publish data, regis ter new hypotheses , support T A 1 with complex team cognitive modeling and T A 3 with experiment des ign

P has e 3 Multiple Humans , S pecialized R oles and S kills

P has e 1

S ingle Human

15 Months

P has e 2

Multiple Humans

15 Months

R efine open-world testbed Develop, implement and refine specialized testbed

Demonstrate agent social awareness that includes human team interactions

Analyze and publish data, regis ter new hypotheses , support T A 1 with team cognitive modeling and T A 3 with experiment des ign

Des ign experiments & measures

P lan and conduct evaluations of AS IS T agents and hybrid teams with challenges of increas ing complexity

Demonstrate agent with social skills in human-machine dyad

R egis ter hypotheses , support T A 1 with cognitive modeling and T A 3 with experiment des ign

Develop and implement open-world testbed

61 70 2

One Perturbation

Three Perturbations

43 5

One Perturbation

Three Perturbations

One Perturbation

Three Perturbations

Figure 2: Schedule of the ASIST program

F. Deliverables

Performers are required to provide, at a minimum, the deliverables described below.

All performers must deliver copies of any technical papers derived from work funded by

ASIST.

Annotated slide presentations must be submitted within one week after the program kickoff meeting and after each program event (site visits, PI meetings, etc.).

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 13

Short quarterly technical reports using a PowerPoint template that summarize key technical accomplishment, list code/results delivered and where, plans for the next quarter, and issues (technical, programmatic, financial) must be provided within 15 calendar days after the end of the quarter.

Monthly financial status reports must be provided within 15 calendar days after the end of each calendar month.

TA1 and TA3 performers are required to deliver software source code prior to each dry-run and evaluation.

Each performer must deliver a final report, at the end of the overall period of performance, which summarizes their project.

G. Government-furnished Property/Equipment/Information

None.

H. Intellectual Property

The program will emphasize creating and leveraging open source technology and architecture.

Intellectual property rights asserted by proposers are strongly encouraged to be aligned with open source regimes. See Section VI.B.1 for more details on intellectual property.

I. Potential Conflicts of Interest

Each proposal may only address a single technical area, but proposers may submit multiple proposals. A proposer may submit proposals for all three technical areas and may participate in either or both TA1 and TA2; however, a proposer selected for TA3 will be unable to perform on any portion of TA1 and/or TA2. This selection process is intended to avoid organizational conflicts of interest (OCI) situations between the research TAs and the integration and evaluation activities, as well as to ensure objective test and evaluation results. The decision as to which proposal to select for award is at the discretion of the Government.

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 14

II. Award Information

A. Awards

DARPA anticipates multiple awards for TA1 and TA2, as well as a single award for TA3. The level of funding for individual awards made under this solicitation has not been predetermined and will depend on the quality of the proposals received and the availability of funds. Awards will be made to proposers whose proposals are determined to be the most advantageous to the Government, all factors considered, including the potential contributions of the proposed work, overall funding strategy, and availability of funding. See Section V for further information.

The Government reserves the right to:

select for negotiation all, some, one, or none of the proposals received in response to this solicitation;

make awards without discussions with proposers;

conduct discussions with proposers if it is later determined to be necessary;

segregate portions of resulting awards into pre-priced options;

accept proposals in their entirety or to select only portions of proposals for award;

fund proposals in increments and/or with options for continued work at the end of one or more phases;

request additional documentation once the award instrument has been determined (e.g., representations and certifications); and remove proposers from award consideration should the parties fail to reach agreement on award terms within a reasonable time or the proposer fails to provide requested additional information in a timely manner.

Proposals selected for award negotiation may result in a procurement contract, grant, OT, or cooperative agreement depending upon the nature of the work proposed, the required degree of interaction between parties, and other factors.

Proposers looking for innovative, commercial-like contractual arrangements are encouraged to consider requesting Other Transactions. To understand the flexibility and options associated with Other Transactions, consult http://www.darpa.mil/work-with-us/contract-management#OtherTransactions.

In accordance with 10 U.S.C. § 2371b(f), the Government may award a follow-on production contract or Other Transaction (OT) for any OT awarded under this BAA if: (1) that participant in the OT, or a recognized successor in interest to the OT, successfully completed the entire prototype project provided for in the OT, as modified; and (2) the OT provides for the award of a follow-on production contract or OT to the participant, or a recognized successor in interest to the OT.

In all cases, the Government contracting officer shall have sole discretion to select award instrument type, regardless of instrument type proposed, and to negotiate all instrument terms and conditions with selectees. DARPA will apply publication or other restrictions, as necessary, if it determines that the research resulting from the proposed effort will present a high likelihood of disclosing performance characteristics of military systems or manufacturing technologies that are unique and critical to defense. Any award resulting from such a determination will include a http://www.darpa.mil/work-with-us/contract-management#OtherTransactions

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 15

requirement for DARPA permission before publishing any information or results on the program. For more information on publication restrictions, see the section below on Fundamental Research.

B. Fundamental Research

It is DoD policy that the publication of products of fundamental research will remain unrestricted to the maximum extent possible. National Security Decision Directive (NSDD) 189 defines fundamental research as follows:

‘Fundamental research’ means basic and applied research in science and engineering, the results of which ordinarily are published and shared broadly within the scientific community, as distinguished from proprietary research and from industrial development, design, production, and product utilization, the results of which ordinarily are restricted for proprietary or national security reasons.

As of the date of publication of this BAA, the Government expects that program goals as described herein may be met by proposers intending to perform fundamental research and does not anticipate applying publication restrictions of any kind to individual awards for fundamental research that may result from this BAA. Notwithstanding this statement of expectation, the Government is not prohibited from considering and selecting research proposals that, while perhaps not qualifying as fundamental research under the foregoing definition, still meet the BAA criteria for submissions. If proposals are selected for award that offer other than a fundamental research solution, the Government will either work with the proposer to modify the proposed statement of work to bring the research back into line with fundamental research or else the proposer will agree to restrictions in order to receive an award.

Proposers should indicate in their proposal whether they believe the scope of the research included in their proposal is fundamental or not. While proposers should clearly explain the intended results of their research, the Government shall have sole discretion to determine whether the proposed research shall be considered fundamental and to select the award instrument type. Appropriate language will be included in resultant awards for non-fundamental research to prescribe publication requirements and other restrictions, as appropriate. This language can be found at http://www.darpa.mil/work-with-us/additional-baa.

For certain research projects, it may be possible that although the research to be performed by a potential awardee is non-fundamental research, its proposed subawardee’s effort may be fundamental research. It is also possible that the research performed by a potential awardee is fundamental research while its proposed subawardee’s effort may be non-fundamental research.

In all cases, it is the potential awardee’s responsibility to explain in its proposal which proposed efforts are fundamental research and why the proposed efforts should be considered fundamental research.

C. Disclosure of Information and Compliance with Safeguarding Covered Defense Information Controls

The following provisions and clause apply to all solicitations and contracts; however, the definition of “controlled technical information” clearly exempts work considered fundamental research and therefore, even though included in the contract, will not apply if the work is fundamental research.

http://www.darpa.mil/work-with-us/additional-baa

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 16

DFARS 252.204-7000, “Disclosure of Information” DFARS 252.204-7008, “Compliance with Safeguarding Covered Defense Information Controls” DFARS 252.204-7012, “Safeguarding Covered Defense Information and Cyber Incident Reporting”

The full text of the above solicitation provision and contract clauses can be found at http://www.darpa.mil/work-with-us/additional-baa#NPRPAC.

Compliance with the above requirements includes the mandate for proposers to implement the security requirements specified by National Institute of Standards and Technology (NIST) Special Publication (SP) 800-171, “Protecting Controlled Unclassified Information in Nonfederal Information Systems and Organizations” (see https://doi.org/10.6028/NIST.SP.800-171r1) that are in effect at the time the BAA is issued.

For awards where the work is considered fundamental research, the contractor will not have to implement the aforementioned requirements and safeguards; however, should the nature of the work change during performance of the award, work not considered fundamental research will be subject to these requirements.

http://www.darpa.mil/work-with-us/additional-baa#NPRPAC https://doi.org/10.6028/NIST.SP.800-171r1

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 17

III. Eligibility Information

A. Eligible Applicants

DARPA welcomes engagement from all responsible sources capable of satisfying the Government's needs, including academia (colleges and universities); businesses (large, small, small disadvantaged, etc.); other organizations (including non-profit); other entities (foreign, domestic, and government); FFRDCs; minority institutions; and others.

DARPA welcomes engagement from non-traditional sources in addition to current DARPA performers.

1. Federally Funded Research and Development Centers (FFRDCs) and Government Entities

a. FFRDCs FFRDCs are subject to applicable direct competition limitations and cannot propose to this BAA in any capacity unless they meet the following conditions: (1) FFRDCs must clearly demonstrate that the proposed work is not otherwise available from the private sector. (2) FFRDCs must provide a letter on official letterhead from their sponsoring organization citing the specific authority establishing their eligibility to propose to Government solicitations and compete with industry, and their compliance with the associated FFRDC sponsor agreement’s terms and conditions. This information is required for FFRDCs proposing to be awardees or subawardees.

b. Government Entities Government Entities (e.g., Government/National laboratories, military educational institutions, etc.) are subject to applicable direct competition limitations. Government entities must clearly demonstrate that the work is not otherwise available from the private sector and provide written documentation citing the specific statutory authority and contractual authority, if relevant, establishing their ability to propose to Government solicitations. This information is required for Government Entities proposing to be awardees or subawardees.

c. Authority and Eligibility At the present time, DARPA does not consider 15 U.S.C. § 3710a to be sufficient legal authority to show eligibility. While 10 U.S.C.§ 2539b may be the appropriate statutory starting point for some entities, specific supporting regulatory guidance, together with evidence of agency approval, will still be required to fully establish eligibility. DARPA will consider FFRDC and Government entity eligibility submissions on a case-by-case basis; however, the burden to prove eligibility for all team members rests solely with the proposer.

2. Foreign Participation Non-U.S. organizations and/or individuals may participate to the extent that such participants comply with any necessary nondisclosure agreements, security regulations, export control laws, and other governing statutes applicable under the circumstances.

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 18

B. Organizational Conflicts of Interest

FAR 9.5 Requirements In accordance with FAR 9.5, proposers are required to identify and disclose all facts relevant to potential OCIs involving the proposer’s organization and any proposed team member (subawardee, consultant). Under this Section, the proposer is responsible for providing this disclosure with each proposal submitted to the BAA. The disclosure must include the proposer’s, and as applicable, proposed team member’s OCI mitigation plan. The OCI mitigation plan must include a description of the actions the proposer has taken, or intends to take, to prevent the existence of conflicting roles that might bias the proposer’s judgment and to prevent the proposer from having unfair competitive advantage. The OCI mitigation plan will specifically discuss the disclosed OCI in the context of each of the OCI limitations outlined in FAR 9.505-1 through FAR 9.505-4.

Agency Supplemental OCI Policy In addition, DARPA has a supplemental OCI policy that prohibits contractors/performers from concurrently providing Scientific Engineering Technical Assistance (SETA), Advisory and Assistance Services (A&AS) or similar support services and being a technical performer.

Therefore, as part of the FAR 9.5 disclosure requirement above, a proposer must affirm whether the proposer or any proposed team member (subawardee, consultant) is providing SETA, A&AS, or similar support to any DARPA office(s) under: (a) a current award or subaward; or (b) a past award or subaward that ended within one calendar year prior to the proposal’s submission date.

If SETA, A&AS, or similar support is being or was provided to any DARPA office(s), the proposal must include:

The name of the DARPA office receiving the support;

The prime contract number;

Identification of proposed team member (subawardee, consultant) providing the support; and An OCI mitigation plan in accordance with FAR 9.5.

Government Procedures In accordance with FAR 9.503, 9.504 and 9.506, the Government will evaluate OCI mitigation plans to avoid, neutralize or mitigate potential OCI issues before award and to determine whether it is in the Government’s interest to grant a waiver. The Government will only evaluate OCI mitigation plans for proposals that are determined selectable under the BAA evaluation criteria and funding availability.

The Government may require proposers to provide additional information to assist the Government in evaluating the proposer’s OCI mitigation plan.

If the Government determines that a proposer failed to fully disclose an OCI; or failed to provide the affirmation of DARPA support as described above; or failed to reasonably provide additional information requested by the Government to assist in evaluating the proposer’s OCI mitigation plan, the Government may reject the proposal and withdraw it from consideration for award.

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 19

C. Cost Sharing/Matching

Cost sharing is not required; however, it will be carefully considered where there is an applicable statutory condition relating to the selected funding instrument (e.g., OTs under the authority of 10 U.S.C. § 2371).

For more information on potential cost sharing requirements for Other Transactions for Prototype, see http://www.darpa.mil/work-with-us/contract-management#OtherTransactions.

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 20

IV. Application and Submission Information

A. Address to Request Application Package

This document contains all information required to submit a response to this solicitation. No additional forms, kits, or other materials are needed except as referenced herein. No request for proposal (RFP) or additional solicitation regarding this opportunity will be issued, nor is additional information available except as provided at the Federal Business Opportunities website (https://www.fbo.gov), the Grants.gov website (https://www.grants.gov/), or referenced herein.

B. Content and Form of Application Submission

1. Abstracts Proposers are highly encouraged to submit an abstract in advance of a proposal to minimize effort and reduce the potential expense of preparing an out of scope proposal. The abstract provides a synopsis of the proposed project, including brief answers to the following questions:

What is the proposed work attempting to accomplish or do?

What is the technical approach (concisely), what are the major innovations, and what are the risks?

How much will it cost, and how long will it take?

DARPA will respond to abstracts with a statement as to whether DARPA is interested in the idea. If DARPA does not recommend the proposer submit a full proposal, DARPA will provide feedback to the proposer regarding the rationale for this decision. Regardless of DARPA’s response to an abstract, proposers may submit a full proposal. DARPA will review all conforming full proposals using the published evaluation criteria and without regard to any comments resulting from the review of an abstract.

Abstract Format: Abstracts shall not exceed a maximum of three (3) pages including the cover sheet and all figures, tables, and charts. The page limit does not include a submission letter (optional).

As a reminder, each abstract submitted in response to this BAA shall address only one TA.

Organizations may submit multiple abstracts to any one TA, or they may submit abstracts to multiple TAs.

All pages shall be formatted for printing on 8-1/2 by 11 inch paper with 1-inch margins and font size not smaller than 12 point. Font sizes of 8 or 10 point may be used for figures, tables, and charts. Document files must be in .pdf, .odx, .doc, .docx, .xls, or .xlsx formats.

Submissions must be written in English. All pages should be numbered.

Abstracts must include the following components:

Cover Sheet: Provide the administrative and technical points of contact (name, address, phone, email, lead organization). Include the BAA number, title of the proposed project, primary subcontractors, estimated cost, duration of the project, and the label “Abstract.”

https://www.fbo.gov/ https://www.grants.gov/

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 21

Goals: Describe what is being proposed and what difference it will make (qualitatively and quantitatively) if successful. Describe the innovative aspects of the project in the context of existing capabilities and approaches, clearly delineating the relationship of this work to any other projects from the past and present.

Technical Plan: Outline and address all technical challenges inherent in the approach and possible solutions for overcoming potential problems. Provide appropriate specific milestones (quantitative, if possible) at intermediate stages of the project to demonstrate progress.

Capabilities/Management Plan: Provide a brief summary of expertise of the team, including subcontractors and key personnel. Identify a principal investigator for the project and include a description of the team’s organization including roles and responsibilities. Describe the organizational experience in this area, existing intellectual property required to complete the project, and any specialized facilities to be used as part of the project. List Government-furnished property, facilities, or data assumed to be available.

Statement of Work, Cost and Schedule: Provide a Rough Order of Magnitude (ROM) cost estimate for resources over the proposed timeline of the project, broken down by year. Include labor, materials, a list of deliverables and delivery schedule. Provide cost estimates for each subcontractor (may be a ROM).

2. Proposals Proposals consist of Volume 1: Technical and Management Proposal (including mandatory Appendix A and optional Appendix B); Volume 2: Cost Proposal; the Level of Effort Summary by Task Excel spreadsheet; and the PowerPoint summary slide.

All pages shall be formatted for printing on 8-1/2 by 11-inch paper with 1-inch margins, single-line spacing, and a font size not smaller than 12 point. Font sizes of 8 or 10 point may be used for figures, tables, and charts. Document files must be in .pdf, .odx, .doc, .docx, .xls, or .xlsx formats. Submissions must be written in English. All pages of Volume 1 should be numbered.

A summary slide of the proposed effort, in PowerPoint format, should be submitted with the proposal. A template slide is provided as an attachment to the BAA. Submit this PowerPoint file in addition to Volumes 1 and 2 of your full proposal, and the Level of Effort Summary by Task Excel spreadsheet. This summary slide does not count towards the total page count.

Reminder – Each proposal submitted in response to this BAA shall address only one TA.

Organizations may submit multiple proposals to any one TA, or they may propose to multiple TAs.

Proposals not meeting the format prescribed herein may not be reviewed.

HR001119S0034 ARTIFICIAL SOCIAL INTELLIGENCE FOR SUCCESSFUL TEAMS (ASIST) 22

a. Volume 1: Technical and Management Proposal The maximum page count for Volume 1 is twenty-five (25) pages, including all figures, tables and charts but not including the cover sheet, table of contents, Statement of Work (see below), Schedule and Milestones, or appendices. A submission letter is optional and is not included in the page count. Appendix A does not count against the page limit and is mandatory. Appendix B does not count against the page limit and is optional. Additional information not explicitly called for here must not be submitted with the proposal, but may be included in the bibliography in Appendix B. Such materials will be considered for the reviewers’ convenience only and not evaluated as part of the proposal.

Volume 1 must include the following components:

i. Cover Sheet: Include the following information.

Label: “Proposal: Volume 1” BAA number (HR001119S0034) Technical Area Proposal title…