HR001120S0028.pdf

Broad Agency Announcement

Air Combat Evolution (ACE) Technical Area 1: Build Combat

Autonomy

STRATEGIC TECHNOLOGY OFFICE

HR001120S0028

March 6, 2020

TABLE OF CONTENTS

I. PART I: OVERVIEW INFORMATION

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

A. Program Overview B. Program Metrics

II. Award Information A. General Award Information B. Fundamental Research

III. Eligibility Information A. Eligible Applicants B. Organizational Conflicts of Interest C. Cost Sharing/Matching D. Other Eligibility Criteria

IV. Application and Submission Information A. Address to Request Application Package B. Content and Form of Application Submission

V. Application Review Information A. Evaluation Criteria B. Review of Proposals

VI. Award Administration Information A. Selection Notices and Notifications B. Administrative and National Policy Requirements C. Reporting D. Electronic Systems

VII. Agency Contacts VIII. Other Information

IX. APPENDIX 1: PROPOSAL SLIDE SUMMARY

X. APPENDIX 2: VOLUME 1 COVER SHEET TEMPLATE

XI. APPENDIX 3: VOLUME 2 COVER SHEET, CHECKLIST AND SAMPLE TEMPLATES

XII. APPENDIX 4: TA-2, TA-3, AND TA-4 REFERENCE INFORMATION

1) TA-2: Build and calibrate trust for air combat local behaviors

2) TA-3: Scale performance & trust to global behaviors

3) TA-4: Build full-scale, combat-representative aircraft experimentation infrastructure

I. PART I: OVERVIEW INFORMATION

Federal Agency Name – Defense Advanced Research Projects Agency (DARPA), Strategic Technology Office (STO) Funding Opportunity Title – Air Combat Evolution Technical Area 1: Build Combat

Autonomy Announcement Type – Initial Announcement Funding Opportunity Number – HR001120S0028 Catalog of Federal Domestic Assistance Numbers (CFDA) – Not Applicable Dates o Posting Date: March 6, 2020 o Proposer’s Day: March 26, 2020 o Questions Due Date: March 31, 2020 by 5:00 PM, Eastern Time (ET) o Proposal Due Date and Time: April 30, 2020 by 5:00 PM (ET)

The Defense Advanced Research Projects Agency (DARPA) is soliciting innovative proposals for the Air Combat Evolution (ACE) Program Technical Area 1 (TA-1): Building Combat Autonomy to increase warfighter trust in combat autonomy by automating aerial within-visual-range (WVR) maneuvering using realistic platforms. The program, ACE, is a vital part of the Mosaic Warfare end-state vision. Proposed solutions should develop innovative approaches that enable revolutionary advances in science, devices, or systems.

Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice.

Total amount anticipated to be awarded – Up to $25 million Multiple awards are anticipated.

Types of instruments that may be awarded – Procurement Contract or Other Transaction Agency Contact

The BAA Coordinator for this effort can be reached at:

HR001120S0028@darpa.mil

DARPA/STO

ATTN: HR001120S0028

675 North Randolph Street Arlington, VA 22203-2114 mailto:HR001120S0028@darpa.mil

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

This publication constitutes a Broad Agency Announcement (BAA) as contemplated in Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016 and 2 CFR § 200.203. Any resultant award negotiations will follow all pertinent law and regulation, and any negotiations and/or awards for procurement contracts will use procedures under FAR 15.4, Contract Pricing, as specified in the

BAA.

The Defense Advanced Research Projects Agency (DARPA) is soliciting innovative proposals for the Air Combat Evolution (ACE) Program Technical Area 1 (TA-1): Building Combat Autonomy.

Technical Areas 2, 3, and 4 were solicited under a previous BAA (HR001119S0051, Amendment 2).

The ACE program will increase warfighter trust in combat autonomy by automating aerial within-visual-range (WVR) maneuvering, colloquially known as a dogfight, using progressively more realistic platforms (following demonstrations in modeling and simulation, the program will graduate to live small-scale unmanned aerial vehicles, and culminate on live operationally representative aircraft). The ACE TA-1 BAA specifically solicits the development of artificial intelligence (AI) algorithms that can conduct aerial dogfighting. A dogfight represents the progression in autonomy from current physics-based automation generally trusted by operators to more complex human-machine collaboration necessary to realize the promise of future manned/unmanned teaming.

A. Program Overview The overarching ACE program will increase trust in combat autonomy by using human-machine collaborative dogfighting as its challenge problem, which also serves as a representation of an entry point into complex human-machine collaboration. ACE will apply AI technologies to the dogfight problem in experiments of increasing realism. In parallel, ACE will implement methods to measure, calibrate, increase, and predict human trust in combat autonomy performance. Finally, the program will scale the tactical application of automating a dogfight to more complex, heterogeneous, multi-aircraft, operational level simulated scenarios informed by live data, laying the groundwork for future live, campaign-level Mosaic Warfare experimentation.

In a future air domain fight contested by adversaries, a single human pilot will be able to increase lethality by effectively orchestrating multiple semi-autonomous, intelligent unmanned platforms from within the manned aircraft. This shifts the human role from sole operator to mosaic system mission commander. In particular, ACE aims to deliver a capability that enables pilots to attend to a broader, more global air command mission while their aircraft and teamed unmanned systems are engaged in individual tactics (see Figure 1). ACE creates a hierarchical framework for autonomy in which higher-level cognitive functions (e.g., developing an overall engagement strategy, selecting and prioritizing targets, determining best weapon or effect, etc.) may be performed by the human, but lower-level autonomy (i.e., details of aircraft maneuver and engagement tactics) is left to the autonomous system. For this to be possible, the pilot must trust combat autonomy to conduct complex behaviors in scenarios such as the WVR dogfight.

Human teaming requires collaborative interaction with calibrated levels of trust and reliance between human and machine agents. Optimally, human-machine teams form a relationship in which the human(s) in or on the loop perform tasks that require human cognition decision-making (to include legal, moral, and ethical decisions [LME]). In these human-machine teams, the autonomous counterparts perform tasks that can be executed equivalently or better than humans while reducing risks to human life without compromising mission effectiveness or ethical considerations.

ACE Approach and Challenges Within the domain of complex autonomous systems, the required operational and strategic behaviors involve high complexity and uncertainty, therefore requiring a less-bounded, highly dynamic AI capability. Highly unconventional operational behaviors may emerge from the autonomous systems that are unexpected by human operators, leading to a sense of uncertainty and lack of trust. As a result, AI for complex problem sets is largely relegated to modeling and simulation (M&S) with no satisfactory method to transition to the real world. For humans to start accepting these systems fully, trust should be established and expanded through bounded, transparent, and predictable behaviors while still preserving the potential for future novelty and situation adaptation. Only after the human-machine tactics co-evolve will the AI be encouraged to move beyond bounded, predictable behaviors, realizing its true potential for improved performance in an increasingly complex battlefield.

ACE will develop performance and trust in combat autonomy using the same bottom-up approach currently employed to build performance and trust with inexperienced human pilots. The ACE program plans to stress dogfight algorithms by applying them to missions of increasing difficulty and realism, similar to human pilots. The dogfight provides human pilots the opportunity to prune output behavior, enhance performance, and calibrate trust for more complicated future combat scenarios such as defensive counter air (DCA) or suppression of enemy air defenses (SEAD).

The ACE program plans to capitalize on AI research that recently demonstrated superhuman performance in applications of increasing complexity, including gaming and self-driving vehicles.

Real-time strategy (RTS) algorithms have exceeded the publicized successes of sequential, perfect-information board games, and are reaching maturity levels that make them well suited to transition to live air combat scenarios. In comparison to RTS games, WVR air combat is a relatively closed problem. While highly nonlinear in behavior, the dogfight benefits from clearly defined objectives:

limited weapon/sensor integration, measurable outcomes, and physics constrained by aircraft

Figure 1. In a “Mosaic” future, humans will predominantly fill the role of battle manager, requiring trust in the combat autonomy and capability provided by unmanned systems dynamics. This “bottom-up” approach is illustrated in Figure 2. The ACE program will move combat autonomy out of the realm of simple predictable behaviors (maneuvers) and into those of increasing complexity.

Figure 2 illustrates the distinction between local and global behaviors with each layer relying on the ones above and below. Current combat autonomy is relegated to the lowest layer, but future Mosaic Warfare needs autonomy throughout all the layers.

The technology development on the ACE program addresses four primary challenges:

1. Increase automated air combat performance in local (individual, team tactical) behaviors;

2. Build and calibrate trust in air combat local behaviors;

3. Scale performance & trust to global (heterogeneous multi-aircraft) behaviors; and

4. Build full-scale air combat experimentation infrastructure.

Program Structure The ACE program is constructed to address four primary technical challenges:

1. Technical Area 1: Build combat autonomy for local (individual, team tactical) behaviors;

2. Technical Area 2: Build and calibrate trust in air combat local behaviors;

3. Technical Area 3: Scale performance/trust to global (heterogeneous multi-aircraft) behavior;

4. Technical Area 4: Build full-scale air combat experimentation infrastructure.

Figure 2. ACE will elevate the current state of physics-based maneuver automation into nonlinear local behaviors and then scale that capability to theater air combat

Technical Areas 2, 3, and 4 (TA-2, TA-3, TA-4) were solicited under a previous BAA (HR001119S0051, Amendment 2), and information regarding TA-2, TA-3, and TA-4 is provided in this BAA for informational and contextual purposes only. Under this BAA, DARPA is soliciting innovative proposals for Technical Area 1 (TA-1) only. Figure 3 highlights the relationship between the different ACE program Technical Areas.

A Government-led independent Experimentation Integration Team (EIT), facilitated by Johns Hopkins Applied Physics Laboratory, will coordinate the interdependent research activities associated with the different ACE Program Technical Areas. Due to the complex integration requirements across multiple performer teams relying on a common experimentation infrastructure, the EIT will be responsible for coordination of the necessary interface control documents (ICD) and application programming interfaces (API) associated with M&S, sub-scale, and full-scale assessments.

Figure 3. Relationships between the ACE Technical Areas

1. Build Air Combat Autonomy for Local (Individual, Team Tactical) Behaviors (as solicited by this BAA)

Technical Area 1 (TA-1) seeks to advance the state of artificial intelligence technologies applied to individual and team tactical behaviors in autonomous air combat maneuvering applications through development and demonstration of autonomous platform maneuvering algorithms in each of the ACE Program’s three Phases. As depicted in Figure 4, implementation will begin in modeling and simulation (M&S) and graduate to progressively more realistic employment including live sub-scale unmanned aerial vehicles (UAVs), and ultimately full-scale combat-representative aircraft modified by the TA-4 Performer.

Figure 4. TA-1 performers to demonstrate WVR combat autonomy in M&S, with sub-scale unmanned aircraft, and with full-scale combat representative aircraft

DARPA anticipates selecting multiple TA-1 performers for funding to:

Develop air combat maneuvering algorithms for individual and team tactical behaviors:

WVR air combat maneuvering engagements – colloquially referred to as dogfights – serve as the application for TA-1 platform maneuvering algorithms. The conditions of each engagement will vary widely, to include 1v1, 2v1*, and 2v2 engagements against adversaries with a broad spectrum of performance and with initial conditions ranging from highly advantageous to highly disadvantageous. Proposers should provide a detailed description of the technical approach, including:

o Structure and type of algorithm (may propose multiple algorithmic approaches) including a description of anticipated implementation.

o Method for training each algorithm (including approach, limitations, time, HW/SW/data requirements, etc.).

o Discussion of expected advantages / disadvantages / limitations for each proposed algorithmic approach.

o Strategy for extending algorithm to compete with adversary fighters with varying performance levels.

o Emphasize any particularly novel elements of the approach.

Proposers must address how their algorithms will be designed and developed to conduct

* Throughout this document we adopt the standard nomenclature for the engagement ordering (aka 1v1, 2v1, 2v2) and present all relationships in the format of the number of friendly forces (traditionally referred to as Blue forces) to the number of adversary or opposition forces (traditionally referred to as Red forces) such that 2v1 represents 2 Blue aircraft versus 1 Red aircraft.

1v1 engagements, and then modified to accommodate 2v1 and 2v2 engagements.

Integrate algorithms with M&S environment, sub-scale live aircraft, and full-scale live aircraft: TA-1 performers will be responsible for integrating their air combat maneuvering algorithms with the EIT-provided M&S environment and sub-scale live aircraft as well as instrumented combat aircraft provided by the TA-4 performer. In each progression (M&S, sub-scale, and live), engagements will begin with 1v1 and progress to 2v1 and 2v2 teamed engagements. Proposers should address how their algorithms will be designed and developed to conduct engagements exclusively in M&S and then be modified to accommodate new requirements in sub-scale, and full-scale implementation. A justification for an algorithmic approach should include the following:

o References to existing applications of same or similar technical approach;

o Previous experience with similar or related efforts especially to the proposed algorithmic approach;

o Identification of Key Personnel engaged in algorithm development, including any relevant technical credentials; and o Any assumptions regarding flight control stated clearly in the proposal response;

different levels of flight controllers may be proposed in M&S, sub-scale, and full-scale portions of the program.

Implement a strategy for interacting the combat autonomy with the human pilot: TA-1 performers will work with the TA-2 performer to determine what information can be made available from the TA-1 algorithm to inform trust assessments, and how critical information should be presented to the human via the TA-2 performer-developed HMI to allow the pilot to interact with and operate the combat autonomy. TA-1 performers must also coordinate control hand-offs with human pilots in support of Trust Assessment Evaluations.

Characterize contribution of tactical behavior algorithms to battle management tasks and provide interface to TA-3 algorithms: TA-1 performers will work with TA-3 performer(s) to resolve a strategy for integrating TA-1 performer-developed local behavior algorithms with TA-3 algorithms performing global behaviors in battle management tasks. TA-1 performers are responsible for making recommendations on how their algorithms could be used in each of the TA-3 tasks and for packaging and releasing their algorithms to TA-3 performers such that they can be integrated with TA-3 algorithms.

This section contains additional details that TA-1 proposers are encouraged to consider and address in their proposals, including:

1. State and Action Spaces

2. Overview of Experimental Testbeds

a. Modeling and Simulation Environment Overview

b. Sub-Scale Live Aircraft Testbed Overview

c. Full-Scale Live Aircraft Testbed Overview

3. Scenario Configuration Parameters

State and Action Spaces

Among the most important details to consider in developing an algorithm capable of performing autonomous platform maneuvering are the system’s state and action spaces; that is, the set of information that can be observed from the system and its environment, and the mechanism for commanding system actions, respectively. The elements of the state space for TA-1 algorithms will be determined based on which properties can be measured and provided to the algorithm in each of the simulated and live aircraft environments, as well as the specific objectives of each evaluation event. Proposers are encouraged to consider the set of state space elements included in Table 1 as representative of the sort of information that will be provided by the EIT, some of which are expected to be available for all platforms and some are expected to only be available for Blue (friendly) platforms. Naturally, we anticipate that the details of these properties, their formats, and how the TA- 1 performer will interact with them will evolve throughout the program and will be refined during Integration Quarterly meetings provided in Table 2 later in the text. Table 1 is provided for reference to aid in the production of a proposal and performers are not limited to the properties contained in it.

Property Object Platform(s) Time since start of engagement N/A Latitude/longitude/altitude All Location relative to scenario center (x/y/z) All Orientation (roll//pitch/heading) All Velocity (x/y/z) All Angular rate (roll/pitch/yaw rates) All Acceleration (x/y/z) Friendly Only Angular acceleration (roll/pitch/yaw second order rates)

Friendly Only

Angle of attack Friendly Only Sideslip angle Friendly Only Current control surface deflections Friendly Only Throttle position Friendly Only Fuel state Friendly Only Current thrust Friendly Only Distances to other aircraft All Angle of bearing to other aircraft All

Table 1. Departure Point for State Space Properties for TA-1 Algorithms

Other derived properties may also be made available for post-mission analysis, such as information associated with the “score” of the engagement, which will inform performance evaluations. Scoring criteria will be based on a notional weapon engagement zone for an air-to-air weapon. The aircraft that emerges from the engagement with a greater score will be awarded a win, and the primary scoring metric will be the win probability of TA-1 algorithms under each scenario configuration.

Proposers should expect engagements where the content or quality of state space measurements is degraded, to represent tactical situations characterized by imperfect sensing conditions. TA-1 proposers should be prepared to address engagements where this degradation is introduced artificially, such that certain properties are unavailable or contain noisy information for the duration of the engagement, and situations where quality of sensor measurements varies with simulated sensor models and the geometry of the fight. In the latter case, sensor model descriptions will be provided to performers to inform strategies for promoting observability of adversary platforms, and compensating for sensing measurement errors. TA-1 proposers are encouraged to articulate in their proposals strategies for developing algorithms that are resilient to partially observable system and environment states, as well as state spaces containing noisy or imperfect information.

The action space selected for TA-1 algorithms will be principally driven by the design of the flight control systems of the instrumented live sub-scale and full-scale aircraft. Performers should expect the action space of their algorithm to contain the following elements for each of the one or two friendly aircraft:

Commanded roll rate Commanded pitch rate Commanded yaw rate Commanded speed brake setting Commanded throttle input

TA-1 proposers are encouraged to specify any recommended action space abstractions other than those specified in this BAA (e.g., waypoint-driven control) as well as proposals for how an abstraction layer may be implemented in M&S and with sub- and full-scale live aircraft to make the preferred interface available.

The process of arriving at final state and action space specifications will be facilitated by the EIT in consultation with the Government. TA-1 proposers should specify in their proposals any additional information that they believe should appear in the state or action space, a rationale for why that information is important, and an explanation of the sensitivity of the algorithm’s performance to the availability of those additional elements. Proposers should remember that the ultimate objective of ACE is the demonstration of autonomy in full-scale combat representative aircraft, so all state and action spaces must be physically achievable in live flight-testing.

Overview of Experimental Testbeds

TA-1 proposers should articulate a plan for integrating air combat maneuvering algorithms with each of these three testbeds spanning the three program Phases and for managing the transitions between Phases.

A) Modeling and Simulation Environment Overview

A common simulation environment will be provided to all performers for use in the development of their autonomy algorithms. Although performers may use other environments for development, it is required that the environment and aircraft models, as provided, be used as the basis for all algorithms delivered to the Government and integrated for Evaluation Events.

The EIT-distributed M&S environment should be derived from JSBSim, an open source flight dynamics model that can be used to model many different aircraft. Additional information is available at this link: http://jsbsim.sourceforge.net/.

The Government will deliver the simulation with a Python-based application programming interface (API) based on OpenAI Gym. Performers should invoke an instance of the simulation using the provided API. Once instantiated, TA-1 algorithms should execute the simulation by synchronously stepping the flight dynamics models of each aircraft in the environment with a function call that includes the commanded action as an argument and returns the current state space and scoring information. These synchronous stepping will happen at the simulation execution frequency and is designed to keep the simulation executing properly and in a known state and is an artifact of the simulation environment since live analog data is not available. Any expected latency between asynchronous commands being sent in the real vehicle (discussed in greater detail below) and the real vehicle’s response is accounted for in the model itself in simulation and is the reason for the overall Phasing of the ACE program which includes an entire Phase 2 using sub-scale aircraft as risk reduction for the more costly full-scale combat representative aircraft in Phase 3.

The M&S environment will provide a configurable frequency of interaction with the TA-1 algorithm, and proposers are encouraged to include considerations for specifying an interaction frequency in their proposals. The simulation execution frequency will initially be set to 50 Hz and may be modulated to provide a better representation of each of the live aircraft interfaces.

Awardees may use “self-play” to develop their algorithms, but only performance against the adversary aircraft provided during Evaluation Events will be evaluated by the Government.

Performers should expect to develop algorithms capable of controlling multiple different aircraft models in JSBSim, including representations of each of the live aircraft to be flown in the sub-scale and full-scale program phases. Proposals should specify a strategy for developing algorithms that are scalable in that they can perform capably when applied to multiple aircraft characterized by significant differences in flight control systems and performance.

Respondents may use the open source virtual simulation application FlightGear (https://www.flightgear.org/) to investigate the JSBSim flight dynamics models for each simulated aircraft model. FlightGear uses an embedded instance of JSBSim to resolve the aircraft control system and dynamic response to control inputs in virtual (human-in-the-loop) simulation execution.

The EIT-developed M&S environment will also include an option to use FlightGear for cockpit-based visualizations driven by TA-1 algorithm control.

B) Sub-Scale Aircraft Testbed Overview

Algorithms developed under TA-1 will be flown on subscale aircraft similar to the Viper+ shown in Figure 5. The goal of this subscale testing is to reduce risk prior to integration on full-scale aircraft.

Proposers are encouraged to describe how they will utilize subscale testing in their response, including how data obtained through iterative subscale testing will be used to improve performance of the algorithms. Proposers are also encouraged to describe how their algorithms will accommodate variations in flight dynamics between different aircraft of the same type, variations due to mass (i.e.

fuel load), and imperfect observations from physical sensors.

Figure 5. Viper+ remote controlled aircraft (comparable aircraft to be used for subscale testing of TA-1 algorithms).

Source: http://www.bvmjets.com/JetKits/Vipe

The state and action spaces between constructive, subscale, and full-scale aircraft will be as similar as possible, but performers should expect some differences to accommodate the physical differences in the platforms. In general, performers should expect to control subscale aircraft using body-relative pitch, roll, and yaw rates, along with throttle setting and speed brake commands, which is similar to how manned aircraft are flown. Performers should not feel required to utilize these actions but should clearly articulate the performance benefits from alternative approaches and include a discussion of how to utilize rate-based controls. The state vector will be generated by the aircraft platform.

Messages to and from the aircraft will utilize the lmcp message protocol developed by the Air Force Research Laboratory. More information on this protocol is available from GitHub (https://github.com/afrl-rq/LmcpGen). Exact message formats with template interfaces will be provided to the TA-1 performers.

The TA-1 performer is not responsible for sensor integration or development of the test infrastructure for subscale testing. The TA-1 performer is responsible for delivering an algorithm or algorithms compliant with the aircraft interface specified by the EIT and that runs on the hardware installed in the aircraft. The exact computer hardware has not been selected, but proposers should assume that low-SWaP processors such as the NVIDIA Jetson TX2 or Intel NUC will be utilized.

C) Full-Scale Aircraft Testbed Overview

Performers should expect the integration of algorithms onto live aircraft will be an iterative process and should include physical aircraft limitations on their algorithms in proposals. For example, although the TA-4 performers will be responsible for such issues as airworthiness, TA-1 performers will be required to submit information to the TA-4 performer via the EIT to support airworthiness reviews. Further, it is anticipated that the initial physical platform limits provided to the TA-1 performers will be more liberal than the final constraints used in live flight testing. Performers should include a discussion of how they will develop algorithms that both fully exploit the kinematic capabilities of the platform (will be used in constructive evaluation) and which respect safety limitations (will be used in live flight testing).

While full details of instrumented full-scale platform design are not yet available, performers should expect to integrate with such platforms as the Aero L-39 Albatros (Figure 6) and the General Dynamics F-16D (Figure 7). It is anticipated that work conducted under this program will be at the unclassified level; however, proposers should provide cost and security plans commensurate with the handling of International Traffic in Arms Regulations (ITAR) controlled information..

Figure 6. Aero L-39 Albatros (Source: https://en.wikipedia.org/wiki/Aero_L-39_Albatros)

Figure 7. General Dynamics F-16D (Source: stripes.com)

Proposals should include considerations such as how algorithms can be tested in simulation to gain confidence in the safety and effectiveness of live aircraft operations and how algorithms will be developed in such a way that they are applicable to the control of aircraft with vastly different performance capabilities and flight control systems.

Scenario Configuration Parameters

Each TA-1 engagement is characterized by a set of parameters, including but not necessarily limited to the following:

Force structures: 1v1, 2v1, or 2v2 engagements.

Specific aircraft model (M&S) or live aircraft (sub- and full-scale) to be flown: As previously indicated, performers should expect different aircraft models to be used in simulated engagements, and multiple different live aircraft during sub- and full-scale aircraft evaluations.

Composition of constructive simulated, virtual simulated, live sub-scale, and live full-scale assets: Engagements may include entirely constructive assets, live assets, or some mix of constructive, virtual, and live assets.

Engagement initial conditions: Each engagement will be characterized by specified relative positions, orientations, initial velocities, fuel states, and other information.

Variety of opponent: During the course of the ACE program, evaluations will involve competition of TA-1 algorithms against progressively more capable autonomy algorithms developed and distributed by the EIT, algorithms developed by other TA-1 performers, and virtual or live aircraft flown by human pilots.

Observability and communications conditions: The properties and quality of measurements in the state space will be determined based on what information can be measured from the selected testbed and the objectives of the exercise. This will include specification of any sensor or communications models or live systems that will dynamically affect the quality or availability of state information.

Interface to aircraft flight control system: The format and content of the action space will be driven by the aircraft model(s) or live aircraft to be flown in each engagement.

Scoring implementation and metrics to be collected: The scoring implementation will determine the winner that emerges from each dogfight, and metrics will drive performance evaluation based on results from multiple engagements.

Prior to each Evaluation Event, performers will be provided with guidance on which scenario configurations will be applicable to that event.

Multiple awards are anticipated for TA-1. Proposals should include a Phase 1 Base Period with separately priced Options for Phases 2 and 3.

2. Experimentation Integration Team The ACE government-led Experimentation Integration Team (EIT) will be facilitated by Johns Hopkins Applied Physics Laboratory (JHU/APL), University Affiliated Research Center (UARC), and is not being solicited under this BAA.

The EIT will facilitate and coordinate the interdependent research activities associated with the different TAs. Due to the complicated integration requirements across multiple performers relying on a common experimentation infrastructure, the EIT will be responsible for the management of the necessary ICDs and APIs associated with M&S, sub-scale, and full-scale experiments to include identification, development and/or maintenance as applicable. This standardization of the experimentation infrastructure is critical to assess individual TA performers. To enable independent and unbiased performer evaluation, the EIT will utilize standardized M&S environments for both the WVR engagements (TA-1) as well as the larger scale, Mission Commander Scenarios (TA-3). These standardized environments will include the development of scripted adversaries for the unaware, limited-aware, and unlimited-aware opponents in all three Phases (M&S, sub-scale, and full-scale) of the ACE program. Lastly, the EIT will be responsible for the purchase, maintenance, and operation of the commercial sub-scale UAVs used throughout all three Phases of the program (and which are the focus Phase 2).

The EIT will host regular Integration Quarterlies to facilitate cross-TA interaction throughout the program; a notional layout for cross TA interaction is laid out in section I.B.3.d). The EIT will complete the following tasks in support of ACE:

Modeling & Simulation (M&S) Framework Development. Develop an M&S framework with live, virtual, and constructive components to support the development and evaluation of autonomous combat capabilities for WVR maneuvering. This includes providing a WVR air combat simulation environment, sub-scale live aircraft, and full-scale live aircraft interfaces to maneuvering algorithms developed by the TA-1 performers.

Adversary Autonomy Development. Develop autonomy capabilities that serve as the adversary force supporting evaluation of autonomy solutions developed by TA-1 performers at each phase of ACE. This includes adversary autonomy at multiple levels of performance (unopposed, limited opposed, and unlimited opposed) to assess performance of the TA-1 algorithms in the constructive M&S and sub-scale live aircraft phases of the program.

3. ACE Program Phasing The overall ACE program structure is shown in Figure 8. The primary focus of Phase 1 is to develop and demonstrate key capabilities in M&S. Phases 2 and 3 will implement the same in sub-scale and full-scale environments, respectively. The government expects multiple TA1 awards. The number of awards in Figure 8 is notional and subject to change at the Government’s discretion.

Figure8. ACE program schedule and phasing

A) Phase 1 (Base Period, anticipated 12 month Period of Performance) Phase 1 is the Base Period for TA-1 and is anticipated to be 12 months. The Phase 1 TA-1 performers will develop 1v1, 2v1, and 2v2 dogfight algorithms for increasing levels of adversary difficulty, which will be tested in M&S against scripted adversaries, human pilots, and other performers’ algorithms. The EIT will provide the M&S environment, scripted adversary AI, and ICDs/APIs for TA-1 performer integration.

The conclusion of Phase 1 includes Technical Data Reviews (TDRs) for each TA designed to provide the necessary evidence to support evaluation of progress against program metrics and transition into the next Phase. Phase 1 major milestones and deliverables are captured in Table 2.

B) Phase 2 (Option Period, anticipated 16 month Period of Performance) Phase 2 is an option period for TA-1 with an anticipated 16 month Period of Performance after option exercise. Proposers should price this Option Period with the same level of detail as that of the Phase 1 Base Period. Phase 2 will focus on scaling WVR combat algorithms developed in M&S to sub-scale experimentation.

The TA-1 performers will refine and implement WVR algorithms onto sub-scale commercial UAV aircraft. Algorithms will be tested in a live environment against scripted adversaries, human pilots, and other ACE program performers’ algorithms. Algorithms will be progressively evaluated in 1v1, 2v1, and 2v2 engagements. The EIT will provide the range environment and platforms, red adversary AI, and ICDs/APIs for TA-1 performers’ integration and assessment.

The TA-1 performers will provide technical guidance and documentation regarding their algorithms to the TA-3 performers who will leverage the TA-1 algorithms at more complex tactical and strategic levels since TA-3 will be representing TA-1 enabled aircraft in larger scale simulations. The TA-3 performers will conduct analysis of scenarios such as CMD and SEAD informed by collected large force live exercise data and then extend that analysis using TA-1 informed aircraft in large scale simulations of missions beyond just the WVR challenge problem that is at the core of the ACE program.

TDRs will also be conducted at the conclusion of Phase 2. Phase 2 major milestones and deliverables are captured in Table 2.

C) Phase 3 (Option Period, anticipated 16 month Period of Performance)

Phase 3 is an Option Period with an anticipated 16 month Period of Performance after option exercise. Proposers should price this Option Period with the same level of detail as that of the Phase 1 Base Period. Phase 3 will focus on extending the WVR combat algorithms and pilot trust assessments developed in M&S and refined in live sub-scale to full-scale platforms.

TA-1 performers will refine and, with the help of the TA-4 performer, integrate WVR combat algorithms onto full-scale aircraft. The TA-4 performer will be responsible for all aspects of full-scale aircraft operations (to include range scheduling, maintenance, and safety/adversary pilot scheduling). Algorithms will be progressively evaluated in 1v1, 2v1, and 2v2 engagements.

Final TDRs will be conducted at the conclusion of Phase 3. Phase 3 major milestones and deliverables are captured in Table 2 below.

4. Cross-TA Interactions While each of the TAs has independent goals and will be assessed independently, the success of the program requires that the TAs work together closely and collaboratively, as each TA is responsible for a critical element of the overall ACE technology demonstration. To ensure this close collaboration across the TAs, each contractor will be required to execute associate contractor agreements as defined in Appendix 5.

Figure 9 illustrates a notional set of cross-TA data and information exchanges for ACE. These exchanges are notional, and the interface specifics will be refined during the integration quarterly meetings outlined in Table 2. Performers should specify the type of information required to satisfy the exchanges suggested below as well as any additional expected information exchanges not reflected below. Performers should identify any innovative means for providing information exchanges. All information exchanged should not include any proprietary information, and performers should identify proprietary issues that would constrain information exchanges. If issues are identified, performers should explain how they intend to share information in a way that meets the intent shown below.

Figure9. Notional Cross-TA Data and Information Exchanges

In support of Phase 1, the EIT will provide constructive M&S environments to TA-1 and TA-3 performers as well as adversary systems to enable algorithm development and test. The EIT will develop virtual simulation cockpits and oversee the integration of HMIs (for both the Dogfight and Mission Commander tasks) and trust assessment equipment into the cockpits. TA-2 will support this objective to ensure that the resulting experimental setup will support the collection of reliable data across multiple scenarios.

In support of Phase 2, the EIT will instrument sub-scale live aircraft with the equipment required to perform autonomous aircraft control and to integrate the aircraft with the greater live, virtual, constructive (LVC) M&S framework. The EIT will also oversee real-time integration of the cockpit simulators with the sub-scale aircraft to enable a human pilot to experience flight of the sub-scale aircraft via a simulated first-person view.

In support of Phase 3, the TA-4 performer will provide the full-scale live aircraft and work with the TA-1 performers and the EIT to devise a strategy for integrating the TA-1 algorithms with the aircraft flight control system. The TA-2 performer will support the instrumentation of the full-scale aircraft with HMIs and trust assessment equipment to ensure that the resulting configuration will enable collection of reliable data in live flight across multiple scenarios.

5. Schedule and Deliverables

MAC Event 1 Kick-off Kick-off 3 Integration Qtrly EIT will facilitate integration exchange 7 QPR Eval Event 10 Integration Qtrly EIT will facilitate integration exchange 11 M&S Demonstration 1v1, 2v1, 2v2 M&S Eval Event

Ph as e

(B as e)

12 Phase 1 TDR Performance results 14 Phase 2 Kick-off Program Plans w/ Emphasis on Phase 2 15 Integration Qtrly EIT will facilitate integration exchange 16 Design Review Sub-scale algorithm designs 18 Integration Qtrly EIT will facilitate integration exchange 19 QPR Eval Event 21 Integration Qtrly EIT will facilitate integration exchange 25 QPR Eval Event 29 Sub-scale

Demonstration 1v1, 2v1, 2v2 Sub-scale Eval EventPh as e

(O pt io n)

29 Phase 2 TDR Performance results 30 Phase 3 Kick-off Program Plans w/ Emphasis on Phase 3 31 Design Review Full-scale algorithm designs 32 Full-scale 1v1 demo Eval Event 35 QPR Status, risks, plans 38 Full-scale 2v1 demo Eval Event 44 Final Demonstration Eval Event

Ph as e

(O pt io n)

45 Phase 3 TDR Performance results Table 2. ACE program major milestones and expected deliverables

The table above summarizes key events and deliverables. The schedule of TA-1 technical activities will be primarily driven by two series of events: Integration Quarterlies and Evaluation Events.

A) Integration Quarterlies

Integration Quarterlies will be coordinated by the EIT and are intended to facilitate frequent integration of products developed by TA-1 performers with technologies and infrastructure developed by the ACE EIT and performers from other ACE Technical Areas. TA-1 proposers are expected to hold quarterly integration events with each of the EIT, the TA-2 performer, the TA-3 performers, and the TA-4 performer, as applicable and as coordinated by the EIT. TA-1 performers will bear the following responsibilities for interacting and collaborating with each of these organizations.

Responsibilities for integration and collaboration with the EIT:

o Integrate TA-1 algorithms with EIT-developed M&S environment to receive state space updates and apply computed actions.

o Integrate TA-1 algorithms with live sub-scale aircraft testbed to receive state space updates and apply computed actions.

Responsibilities for integration and collaboration with the TA-2 performer:

o Integrate TA-1 algorithms with TA-2 performer-developed HMI and trust evaluation infrastructure per requirements specified by TA-2 performer.

o Devise strategy for aircraft control hand-offs between TA-1 algorithm and human pilot in M&S, sub-scale, and full-scale tests and evaluations.

Responsibilities for integration and collaboration with the TA-3 performers:

o Package TA-1 algorithms for use by TA-3 performers as local behaviors within larger operational exercises orchestrated by TA-3 algorithms.

o Characterize role of TA-1 algorithms within Mission Commander scenarios used in

Trust Assessment Evaluations.

Responsibilities for integration and collaboration with the TA-4 performer:

o Integrate TA-1 algorithms with live full-scale aircraft instrumented and operated by TA-4 performer, including interfaces to sensor and communications systems for state space input and flight control system for computed actions.

o Integrate with TA-2-designed HMI hardware and onboard instance of TA-3-developed Mission Commander scenario for Phase 3 Trust Assessment Evaluations.

B) Evaluation Events

TA-1 evaluation events will occur multiple times within each phase of the ACE program and will involve competitions against EIT-developed adversary algorithms, autonomous agents driven by algorithms developed by other TA-1 performers, and human pilots. These evaluation events will occur in M&S, with sub-scale aircraft, and with combat representative full-scale aircraft. Specific guidance will precede each evaluation event, including:

Evaluation objectives;

System architecture diagrams specifying applicable versions of all interface control documents, software and hardware products developed and distributed by EIT and other ACE performers;

Scenario configuration, or set of scenario configurations to be evaluated (see “Scenario Configuration Parameters” for additional details); and

Any unique deliverable requirements dictated by the intent of the evaluation event.

In addition to any unique deliverable requirements dictated by the intent of the evaluation event, performers are expected to bring the following to each evaluation event:

Executable versions of an autonomous WVR Dogfight agent, delivered prior to each evaluation event.

o Awardees are expected to submit all content required to execute their algorithm against the provided environment (M&S, sub-scale, full-scale) application through its API. Depending on the software design of the algorithm, this may include script files or executables compiled from source code. If algorithms are submitted in a binary format, source code must be submitted as well. DARPA will provide the hardware required to deploy algorithms during the evaluation events.

Details to follow on the specifics of the provided hardware configuration, and awardees should specify whether custom hardware will be required to execute their algorithms during the evaluation event.

o Instructions for installing and running the autonomous agent in the simulation will accompany the executable code for the agent.

Prior to each evaluation event, a technical description of the algorithms used in the event along with a technical description of:

o Algorithmic approach to build the agent (reinforcement learning, genetic fuzzy tree, etc.);

o Training/learning process;

o Training convergence rates, computational requirements, and memory requirements for initial training;

o Computational and memory requirements to run the trained AI system;

o Types and amounts of data required for training;

o Ability to update or extend the AI decision system without complete re-training;

o Ability to adapt “on the fly” to unexpected adversary behavior or capabilities;

o Explainability and understandability of the decision process;

o Ability to replicate and implement known effective tactics;

o Ability to discover novel effective tactics; and o Ability to prove behavioral properties and performance of the trained system.

Performers will work with the EIT and other performer organizations prior to each evaluation event to ensure that their algorithms are prepared for integration with any additional program infrastructure and technologies required to perform the evaluation event.

B. Program Metrics

In order for the Government to evaluate the effectiveness of a proposed solution in achieving the stated program objectives, proposers should note that the Government hereby promulgates the following program metrics that may serve as the basis for determining whether satisfactory progress is being made to warrant continued funding of the program. Although the following program metrics are specified, proposers should note that the Government has identified these goals with the intention of bounding the scope of effort, while affording the maximum flexibility, creativity, and innovation in proposing solutions to the stated problem.

The ACE metrics have been designed to objectively measure the performance of each of the TA performers with increasing fidelity and levels of difficulty in each Phase and facet of the program with threshold (Th) and objective (Ob) levels based on human pilot training criteria. These metrics will provide a basis for determining the success of each phase.

Figure10. ACE program metrics

The program will evaluate performance throughout using the metrics shown above in Figure 10 which are derived from human pilot training criteria (such as Win Probability, Pw).

II. Award Information

A. General Award Information

DARPA anticipates multiple awards. The amount of resources made available under this BAA will depend on the quality of the proposals received and the availability of funds. For planning purposes, the Government has budgeted the following funding levels for awards under this BAA (these amounts are approximate, subject to change, and do not include funding set aside for Government support):

Phase 1 Base Period (12 months), Phase 2 Option Period (16 months), and Phase 3 Option Period (16 months): $25 million

Proposers should include Phase 1 as the Base Period and Phases 2 and 3 as separately priced Options in their proposal.

The Government reserves the right to select for negotiation all, some, one, or none of the proposals received in response to this solicitation and to make awards without discussions with proposers.

The Government also reserves the right to conduct discussions if it is later determined to be necessary. If warranted, portions of resulting awards may be segregated into pre-priced options.

Additionally, DARPA reserves the right to accept proposals in their entirety or to select only portions of proposals for award. In the event that DARPA desires to award only portions of a proposal, negotiations may be opened with that proposer. The Government reserves the right to fund proposals in phases with options for continued work, as applicable.

The Government reserves the right to request any additional, necessary documentation once it makes the award instrument determination.