HR001118S0022-Amendment-01.pdf

Broad Agency Announcement

Amendment 1

Collection and Monitoring via Planning for Active

Situational Scenarios (COMPASS)

STRATEGIC TECHNOLOGY OFFICE

HR001118S0022

April 16, 2018

The purpose of this amendment is to make revisions to language in the following sections, which are highlighted in yellow throughout the document:

1. PART I: OVERVIEW INFORMATION; Dates

2. I. Funding Opportunity Description

3. A. Program Overview; Section 2. Program Description/Scope

4. A. Program Overview; Section 3. Program Structure

5. A. Program Overview; Section 3. Program Structure; e) Test and Evaluation

TABLE OF CONTENTS

PART I: OVERVIEW INFORMATION

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

A. Program Overview B. Program Metrics C. Requested Technical Information D. Use Case: Stability Operations in Atropia

II. Award Information A. General Award Information B. Fundamental Research

III. Eligibility Information A. Eligible Applicants B. Organizational Conflicts of Interest C. Cost Sharing/Matching D. Other Eligibility Criteria

IV. Application and Submission Information A. Address to Request Application Package B. Content and Form of Application Submission

V. Application Review Information A. Evaluation Criteria B. Review of Proposals

VI. Award Administration Information A. Selection Notices and Notifications B. Administrative and National Policy Requirements C. Reporting D. Electronic Systems

VII. Agency Contacts VIII. Other Information IX. References

X. APPENDIX 1: PROPOSAL SLIDE SUMMARY

XI. APPENDIX 2: FOR OFFICIAL USE ONLY (FOUO) DOCUMENTS REQUEST FORM

XII. APPENDIX 3: VOLUME 1 COVER SHEET TEMPLATE

XIII. APPENDIX 4: VOLUME 2 COVER SHEET, CHECKLIST AND SAMPLE TEMPLATES ..53

PART I: OVERVIEW INFORMATION

Federal Agency Name – Defense Advanced Research Projects Agency (DARPA), Strategic Technology Office (STO) Funding Opportunity Title – Collection and Monitoring via Planning for Active

Situational Scenarios (COMPASS) Announcement Type – Initial Announcement Funding Opportunity Number – HR001118S0022 Catalog of Federal Domestic Assistance Numbers (CFDA) – Not Applicable Dates o Posting Date April 4, 2018 o Proposer’s Day – March 30, 2018 o Proposal Abstract Due Date and Time – April 20, 2018, 4:00PM Eastern o Questions Due Date – April 26, 2018 o Proposal Due Date and Time – June 4, 2018, 4:00PM Eastern o BAA Closing Date and Time – September 26, 2018, 4:00PM Eastern

Number of Awards – Multiple awards anticipated for all TAs.

Types of instruments that may be awarded – Procurement contract or other transaction Agency Contact o The BAA Coordinator for this effort may be reached at:

HR001118S0022@darpa.mil

DARPA/STO

ATTN: HR001118S0022

675 North Randolph Street Arlington, VA 22203-2114 mailto:HR001118S0022@darpa.mil

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

This publication constitutes a Broad Agency Announcement (BAA) as contemplated in Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016 and 2 CFR § 200.203. Any resultant award negotiations will follow all pertinent law and regulation, and any negotiations and/or awards for procurement contracts will use procedures under FAR 15.4, Contract Pricing, as specified in the

BAA.

The Defense Advanced Research Projects Agency (DARPA) is soliciting innovative proposals for the Collection and Monitoring via Planning for Active Situational Scenarios (COMPASS) program. COMPASS develops decision aids that drive probing actions designed to improve situational awareness and reduce the ambiguity of actors and objectives in gray zone environments.

Gray zone warfare is characterized by “limited conflict, sitting between ‘normal’ competition between states and what is traditionally thought of as war” 1. Gray zone actors use techniques such as misinformation, intimidation, pressure, and disruptions of services to destabilize nations and possibly produce advantageous conditions for military engagements.

COMPASS decision aids will enhance gray zone operations by helping Joint Task Force Commanders understand the intent and tactics of adversaries as they engage in deceptive and covert gray zone activities. Instead of relying on passive collection of sensory data, COMPASS will employ probing strategies that stimulate the environment with controlled actions to generate desirable intelligence. Examples of probing strategies include targeted information collection activities (such as interaction with the local populace), basic protection services (such as critical infrastructure security), and proactive tasks (such as counter messaging, military presence, or exposure of subversive actors).

COMPASS is soliciting proposals in the following three technical areas (TAs):

1. Technical Area 1: Decision aids that discover the intent of gray zone actors, including goals, objectives and desired strategies;

2. Technical Area 2: Decision aids to estimate the adversary campaigns, including the actors, relationships, timings, and dependencies of the adversary tactics; and

3. Technical Area 3: An architecture that integrates the decision aids and allows operators to assess the decision space, recommend probing actions, monitor progress, and suggest adjustments.

Proposed efforts should make advances in the computational areas of artificial intelligence, game theory, complex and non-linear systems, and human-machine decision making in complex situations under uncertainty and risk. COMPASS will explore the benefits that probing actions can provide to intelligence analysis (e.g. indications and warnings), against the risks that these actions might incur (e.g. escalation) in socio/physical environments characterized by dynamism, complexity, and instability. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice.

1 The full description of all citations is included in the References section at the end of the BAA

A. Program Overview

1. Background

The gray zone is emerging as a dominant conflict space, and gray zone activities are a major source of instability worldwide. Gray zone actions are typically complex, covert, and therefore difficult to detect. These attacks confound the target countries and dramatically advance the political goals of the perpetrator. Such expertly orchestrated attacks involve foreign policy expertise, military strategy, diplomacy, and depending upon the action, economic and/or cyber security expertise.

Gray zone interactions sometimes result in unanticipated or escalatory outcomes. The net results are that military commanders, senior diplomats, and political leaders are left unprepared for conflict.

Several recent publications, such as (Defense Sciense Board, 2016) and (Mazarr, 2016) point out two main challenges in gray zone environments. First, the intent of the adversary is not singular, or not even exclusive of other goals. Adversaries may try to pursue simultaneous goals in multiple spheres of influence, e.g. diplomatic, economic, and military. Moreover, such goals are rarely independent of each other and may be mutually reinforcing. Longer-term goals may be contingent on near-term objectives as the enemy gradually escalates their gray zone campaign. New capabilities are needed that address these inter-dependencies and multiple timescales of enemy intent, in contrast with existing methods that assume simple interactions.

Secondly, the variety of actions an adversary has at its disposal to conduct gray zone warfare is very broad. Gray zone operations do not rely on kinetic operations alone. Rather, they typically involve both physical and social activities, making gray zone warfare difficult to discern.

Moreover, gray zone adversaries are quick to adapt to changing conditions and improvise new interventions. This creates a significant challenge on how to characterize the dynamics of the environment while the dynamics themselves are changing. New automated reasoning methods are needed that go beyond first order state changes and account for higher-order shifts in intent and behavior.

Current military simulations and decision support tools like intelligence and command and control (C2) systems are ineffective because they were not designed to address gray zone threats. Analysis tools like wargames that involve both military and diplomatic experts, analytical tools developed by the intelligence community, and the families of ‘DIME’2 and ‘PMESII’3 modeling frameworks developed by the US military (OSD, 2009) also fall short because of their inability to adapt to unforeseen situations. In a recent wargame simulation conducted by the Center for Strategic and Budgetary Assessments (CSBA), operators lacked the tools to counter gray zone aggression, forcing them to escalate the war to a major military engagement because of their inability to adapt proportionally to the adversary actions (OSD, 2009). New decision aids are required that let decision makers experiment in a virtual environment with different intelligence collection strategies in order to create better informed plans of action.

2 DIME: Diplomatic, Intelligence, Military, and Economic actions 3 PMESII: Political, Military, Economic, Social, Information, and Infrastructure effects.

2. Program Description/Scope COMPASS (Figure 1) aims to develop decision aids technologies that provide a leap-ahead in situation awareness for gray zone environments. COMPASS decision aids will assist military commanders in performing the joint intelligence preparation of the operating environment (JIPOE). According to Joint Pubs 2-0, ‘JIPOE is the continuous decision [making] process through which J-2 manages the analysis and development of products that help the commander and staff understand the complex and interconnected OE—the composite of the conditions, circumstances, and influences that affect the employment of capabilities that bear on the decisions of the commander’ (OSD, 2009).

Traditional military decision-making approaches employ the Observe, Orient, Decide, and Act (OODA) loop construct, a framework to quickly observe and react to unfolding events in order to gain temporal and cognitive advantage. However, the OODA loop construct was developed for kinetic fights, leaving it largely ineffective for gray zone conflict.

Events in the gray zone battlefield are hard to discern because they blend in with the surrounding socio-physical environment. Moreover, any readily available evidence is often implanted by adversaries to generate ambiguity and create confusion. To address these challenges, COMPASS proposes to extend the OODA loop construct by adding a probing sub-loop that stimulates the environment in a targeted way and reveals adversarial actions. As shown in Figure 1, COMPASS will develop tools for both long-term and short-term understanding of the adversary.

Figure 1: COMPASS extends the OODA loop through the probing sub-loop that assesses both the long-term objectives and the short-term tactics of the adversary. The COMPASS toolbox assists Commanders with the

JIPOE process, and the work products feed into the course of action (COA) planning phase.

The long-term assessment process will examine the intent of the adversary as it plans its strategy and sets conditions for its campaign. In this stage of the engagement, the environment is changing slowly, the intensity of the campaign is fairly low and activities occur over months to years (and in some instances decades). Long term assessment is followed by short-term analyses designed to discover tactics the adversary uses to achieve its intent. This includes sequences of actions, the actors involved, and the venues used to perform its campaign. During this stage, the environment is very dynamic and has the potential to deteriorate into instabilities and escalation. COMPASS will develop and integrate these two sets of decision aids into a commander’s toolbox intended to significantly improve overall gray zone situation awareness.

COMPASS information products will inform the Course of Action (COA) planning phase during which blue commanders develop reactive and pro-active campaigns. The Joint Operations Planning Process (which includes mission analysis, COA development, analysis, comparison, and approval) is outside the scope of this program. COMPASS users are typically operations and planning staff that support Joint Task Force commanders helping host nations defend against the subversive actions of gray zone actors. COMPASS focuses on environments vulnerable to the destabilizing effects of adversaries through both kinetic and non-kinetic means, such as urban centers. COMPASS addresses the strategic / operational level of war and provides decisions aids to develop a high fidelity understanding of the adversary’s gray zone campaign. Bidders should review documents such as (Office of the Secretary of Defense, 2009), and (Office of the Secretary of Defense, 2009), for an understanding of the granularity of the events of interest to the program, as well as the probes and the effects they might have on the environment. For a detailed use case scenario, with examples of blue probes and adversary responses, see section I.D.

3. Program Structure As outlined in Figure 2, COMPASS will consist of three technical areas (TAs): TA1, which focuses on long-term adversary strategies of adversary intent; TA2 for short-term situational awareness about the tactics and dynamics of the operating environment; and TA3 for building the COMPASS commanders toolbox. DARPA anticipates multiple awards for all TAs.

Figure 2: COMPASS fuses physical with social data to reduce the ambiguity in gray zone environments. Two technical areas, TA1 and TA2, discover intent and tactis respectively, and are integrated into a unified system via

TA3, the system integrator, which also provides the data management and the user interface

COMPASS is a data analysis program. It will not develop new social-cultural-behavioral models.

Nor will it develop new sensors or exquisite new signal processing algorithms. Instead, proposers should leverage existing state of the art technologies, such as existing advances in event extraction from non-structured source (e.g. topic modeling and event extraction), or modeling frameworks such as the US Army’s Athena stability simulation tool. Proposers may also employ their own specialized models, sensors, and signal processing technologies if those technologies provide improved intelligence and do not require significant additional investment.

COMPASS technologies should not be limited to any single area of operation, or type of adversary.

Proposers need to describe how their systems can be adapted to different scenarios, and how they will be able to handle different types of gray zone situations including, but not limited to, critical infrastructure disruptions, information operations, political pressure, economic extortion, security force assistance, corruption, election interference, social discord and confusion, etc. To cover these types of situations, DARPA expects performers to process data that capture military engagements (including troop movements and exercises), utilities (including critical infrastructure and transportation disruptions), social behavior (including demographics and ethnical characteristics), politics / economy (including government structure, policies, and economic activities), and non-military actor activities (including crime, riots and activism). See section I.A.3.c for a list of data sets of interest to DARPA. The government team will furnish several sets of data for training and T&E purposes, however, proposers may also employ their own datasets, provided they describe the dataset and explain how these datasets benefit their technologies. In addition, proposers need to procure that data and make it available to the T&E for inclusion in the virtual environment.

The COMPASS Test and Evaluation (T&E) team will evaluate technologies in a virtual environment, driven by a live-constructive simulation. This setting allows the government to vary the characteristics of both the data and the situation, and to generate performance curves through multiple experiments. The government will procure and make available to performers data and models widely accepted as representative of the real world. Moreover, human users will participate as live actors in Phase 2 simulation. For more details on the evaluation and simulation approaches, see section I.A.3.d.

Figure 3 shows a high-level functional view of the integrated COMPASS system. Military users express their situational awareness needs via TA3. TA1 and TA2 analyze sensed data and, based on the SA need and the knowledge of the environment, suggest probing actions designed to reduce ambiguity. Users review the recommended actions, and make decisions on which actions to execute. The results of the probing actions become the new state of the OE that TA1 and TA2 analyzes in

SA need

Adversary state of intent

/ goals

Adversary tactics sensors

Key actors?

Proposed probes

Proposed probes

Playbook?

Goals?

Expose

Inform

Assist sensors

State of OE

State of OE

OE: Operating Environment SA: Situational Awareness models models

TA1

TA2

SA need

Commander’s SA needs

(specify & focus)

Commander’s probing actions (review & execute)

Commander’s SA results

(analyze & refine)

TA3

Figure 3: The COMPASS tech areas interact closely to produce an integrated system an iterative fashion to build a high fidelity understanding of the adversary’s gray zone campaign.

For a description of the TA1/TA2 interaction, see section A.3.d. below. This continuous process ensures COMPASS is robust to adversary adaptation and keeps up with changes in the environment. The rest of the section describes each tech area in more detail and highlights the topics that are expected to be addressed as part of this effort. The government provides these topics in an effort to guide the research agenda, however, proposers should emphasize whatever structure makes the most sense from the perspective of their technical approach, both in terms of capabilities and cost.

a) Technical Area 1 – Reveal adversary intent TA1 will ascertain the intent of the adversary in competitive environments where actors make rational decisions to achieve their goals. COMPASS may use a game formulation or artificial intelligence planning techniques to engage the adversary in a ‘back and forth’ exchange of actions and counter-actions that generate evidence pointing to their strategies and end goals. The use of terms such as ‘game formulation’ in this BAA should not be interpreted as prescribing or discouraging any specific modeling or computational approach. It is the specific intent of this BAA to solicit a wide range of technical approaches that infer strategies and policies in the complex environment of gray zone situations.

The inputs to TA1 may include models of the environment and time stamped observations about key actors, roles and relationships. The outputs are ranked lists of both blue and red strategies, with probabilistic assumptions and confidence levels. Blue strategies represent probing actions, and red strategies represent adversary response. More specifically, red strategies capture both the states adversaries intend to achieve (in an n-dimensional space of PMESII effects), as well as overarching playbooks (such as policies in AI planning) used to reach those states. Performers must clearly identify which models they plan to use and how they will apply them to their technical approach. Performers will be evaluated on their ability to solve well-bounded mixed strategy approaches using their proposed algorithms, as well as the ability to improve strategies with learning technologies. In particular, metrics include precision and recall of proposed strategies, information gain with probing actions, convergence of hypotheses and adaptability in repeated engagements. For more details on the metrics see section I.B.

TA1 proposers may address one or more of the following topics:

Support multi-faceted theories of adversarial behavior: Traditional adversarial models explore simple moves and counter-moves between antagonizing actors and anticipate single level effects generated from isolated engagements. In gray zone environments, however, the battlefield is high interconnected and actions in one domain (e.g. cyber) may produce outcomes in a different domain (e.g. economy). They may also result in second and third order effects (such as instability of a government). TA1 performers need to accommodate these complexities in the environment, and account for goal-oriented adversaries. Leveraging and extending existing adversarial theories is acceptable. DARPA is explicitly interested in normative theories that describe how a rational agent will act. Descriptive theories that focus on other intangible aspects such as human judgment, irrationality, biases, cognitive limitations, etc., are outside the scope of this BAA.

Account for adversarial decisions over a large space of possible action with both physical and social characteristics. The range of actions actors can take in a gray zone environment is large and it spans both the physical space of movement and services, as well as the cognitive space of perceptions and disposition. Moreover, these actions can have different types of escalatory impacts, ranging from subversion and disruption of critical infrastructure and territory grabs on the physical domain, to shaping opinion, coercion and social capture of the cognitive domain. TA1 performers are expected to develop technologies that consider this broad range of potential actions and provide mechanisms that estimate the impact they have on the environment. In addition, proposers need to articulate technical approaches that leverage those estimates (along with observable evidence) to drive the probing actions of COMPASS.

Account for actors with diverse, and often conflicting objectives. Gray zone warfare is not linear. Enemies may try to pursue simultaneous goals in multiple spheres of influence, e.g.

diplomatic, economic, and military, and often such goals can be dependent and mutually reinforcing. Failure in one sphere of influence might trigger a shift in a position in another sphere that could upend understanding of their strategy. TA1 proposers need to clearly articulate how they would deal with such a complex environment, and what techniques they will use to infer intent in a computationally efficient manner.

Account for situations where actors adapt their behavior. Gray zone actors can adapt by simply denying involvement or by changing ‘the rules of the game’ such as forming alliances, re-interpreting international norms, and in general making efforts to undermine established processes.

Traditional warfare usually ends when some clear-end goals are satisfied. Winning in the gray zone is elusive since goals do not have well defined measures, and actors continue to adapt as they position for power and make tradeoff decisions between short-term goals and long-term opportunities. In addition, adversaries engage in concealment activities that either hide their actions or change the narrative of the situation to influence the decision-making process of the US and its allies. Technologies in this area need to reason about both immediate benefits as well as repeated engagements where reputation and trust are critical for success. DARPA is particularly interested in reasoning technologies that associate evidence to hypotheses, as opposed to cognitive limitations and human biases.

Proposed approaches may incorporate encoded expert knowledge in forms such as pre-defined sets of actions, states, benefits and observations as an aid to bootstrapping development of the TA1 technologies. TA1 proposals should clearly articulate the types of encoded knowledge to be used, the anticipated sources, and the ways in which their proposed approach uses expert knowledge.

DARPA is interested in learning technologies that refine the model of the adversary based on repeated engagements, observations, empirical evidence, and statistical methods. However, complete reliance on automated methods is unrealistic since human input may be required to resolve areas of significant uncertainty or assist filling in gaps in the models. Proposals should describe how their decision aids will interact with human operators. End user interaction will be via the TA3 commander’s toolbox, which will use well defined application program interfaces (APIs) to interact both with the TA1 models and with the TA1 algorithms.

b) Technical Area 2 – Discover adversary tactics TA2 performers will refine the adversarial intent hypotheses produced by TA1 in order to reveal the actors, relationships, and timing of adversary activities. The purpose of TA2 is to control an adversarial situation by generating actions that fit a ‘reference’ behavior of the environment. Since the environment can be highly unstable, the goal is to probe the environment in the most narrowly prescribed manner that generates valuable intelligence while avoiding undesired behavior. TA2 may achieve this goal by leveraging control theory or other similar constructs. For example, basic control frameworks consist of two main components: a controller and an estimator. The TA2 controller might choose actions that maximize an optimization function, such as discovery of actors, types of relationships, etc. Similarly, the COMPASS estimator might detect activities in the environment indicative of the running hypothesis, including anomalies not fitting a pattern of behavior. The controller might then adjust its control signals based on the new observed behavior.

This interplay between the controller and the estimator would presumably make control-based decision aids robust to complex environments.

TA2 will process TA1 work products (intent and strategies) along with an expanded list of adversary activities, including historical trends, interaction dynamics, and real-time streams of current events. Similar to TA1, models of the environment (utilities, crime, social structure, etc.)

will drive technology development and analysis. TA2 outputs will include blue probing actions and their expected collection value, the projected responses from the adversary with potential for instability and escalation, and intelligence products with tactics the adversary might employ in order to achieve its intent(s). TA2 approaches will be evaluated for their ability to control non-linear complex systems where the dynamics of the system are partially known. Performance metrics are similar to TA1 (section I.B.).

TA2 proposers may address one or more of the following topics:

Develop controller actions that avoid escalation and instabilities. Traditional kinetic warfare follows well-known physical equations. Artillery, maneuver, ISR coverage, attrition, etc.

can all be modeled with equations that are well understood and tested in many previous fights.

Even deception and psychological factors have been explored extensively in tight and intense environments. However, the gray zone domain exhibits emergent behavior. The underlined system contains discontinuities that lead to high uncertainty and complex behavioral patterns (for example population discontent that builds up and eventually leads to major protests and riots). Moreover, the environment is characterized by non-linear dynamics because the actors have their own, diverse, and in some cases competing objectives. DARPA is soliciting technologies that can identify those discontinuities and develop probing actions, while minimizing escalation or other undesirable effects. Technologies such as stability, resilience, anti-fragility, and robustness are of interest to DARPA.

Accommodate highly interconnected and interdependent physical and social environments.

As described in section I.D., the operating environments for COMPASS are highly interconnected and dynamic urban centers. Urban agglomeration is a complex adaptive system characterized by a high degree of density, expressed through complex human-made physical terrain, a population of significant size and varied composition, functional infrastructure, and informational complexity.

Such a system is characterized by multiple, interdependent, dynamic layers, with transport, communications, education, cultural, public safety and utility infrastructure forming a complex pattern of flows, linkages and nodes within an urban matrix. In addition, these systems can interact at different time scales. For example, in a short time scale, malicious actors may exploit the internet to change public opinion. In a longer timeframe, demographic composition, cultural associations, social modernization, etc. shape the receptivity of the population to a gray zone narrative. TA2 performers are expected to develop computational approaches that consider these complex interdependencies and nominate probing actions that drive the system to situations with high degree of observability and discriminative information value.

Estimate behavior in response to the probing actions. One challenge with active operations is determining causal linkages between actions and responses. TA2 solutions need to recognize subtle activities and distinguish evidence resulting from probing actions versus naturally occurring events. To this end, it will be necessary to detect very specific signals from complex, multi-dimensional, and noisy data streams. The signal itself may be noisy and difficult to recognize even if “detected”. Previous approaches to this problem include anomaly detection, causal analysis, and normative behavioral understanding. Performers are expected to clearly describe their approaches, including which sensors, data, and metrics they plan to use.

As described in the TA1 section above, proposed TA2 approaches may incorporate encoded expert knowledge, and use learning technologies to improve results. Moreover, technologies will need to interact with human operators through the TA3 commander’s toolbox. Proposers must describe clearly the interaction process and how operator knowledge and guidance is used by TA2 technologies.

c) TA1/TA2 Interaction

While TA1 and TA2 decision aids, both improve the situational awareness of the gray zone environment, they provide complementary capabilities. By focusing on adversary intent, TA1 attempts to discover the end state that the adversary is trying to achieve. TA1 analyzes the semi-steady aspect of the environment where the complexity and intensity are fairly low, the timelines are longer, and when the adversary is planning its campaign that has the most consequential influence on the events that will follow. The TA1 probing actions operate at the strategic level of warfare. By contrast, TA2 refine this information by determining ‘who’, ‘when’, and ‘where’ will accomplish the intent. It operates in highly dynamic environment where the interactions are highly rapid and where the risk of escalation is significant. As such, TA2 probing actions operate at the operational/tactical level of warfare.

DARPA anticipates that TA1 and TA2 technologies are highly dependent on each other and might interact in an iterative fashion. In a traditional linear mode, hypotheses from TA1 feed into model refinement by TA2. However, as the decision aids refine their output products, more iterative interaction mechanisms are possible. For example, as TA2 develops adversary tactics, it might discover a new pattern of behavior that does not fit any prior hypothesis of intent. This will require additional analysis by TA1 to take into consideration new patterns and generate alternative hypotheses. Alternatively, as TA2 explores and tests different aspects of the adversary model, it might repeatedly interact with TA1 to change the hypotheses in subtle ways, or to modify the certainty score of the produced results.

TA1 and TA2 performers need to articulate the mechanisms (e.g. interfaces or interaction patterns) that allow their decision-aid technologies to be components of a larger commander’s toolbox system as described below.

d) Technical Area 3 – Integrate prototypes into the commander’s toolbox The TA3 performer is responsible for overall system functions and their utility to human operators.

The commander’s toolbox will integrate TA1 and TA2 prototypes into a unified architecture that adapts to a variety of input sources and scenarios and provides the mechanism for users to assess the tradeoff between probing actions and the instabilities they might generate in a non-linear and partially unknown environment. TA3 solutions provide services to ingest and manage input data and to encapsulate and task TA1 and TA2 prototypes. TA3 solutions also provide the physical environment needed to execute experiments, evaluations, and demonstrations. TA3 provides the single touch point between the computational prototypes (from TA1 and TA2) and the human operator. The evaluation team will evaluate the TA3 solution for its ability to process data generated by a medium size urban center and the ability of non-technical users to interact with the system and turn computer generated hypotheses into cognitive assessment of the environment.

Specific metrics are listed in section I.B.

TA3 proposers may address one or more of the following topics:

Develop the Human Machine Interface to TA1 and TA2 sub-systems. The COMPASS system will let operators understand the uncertainty of the operating environment, assess tradeoffs of probing actions, and steer the system towards specific actions. The TA1 and TA2 technologies might potentially generate many possible probing actions and could produce a wide range of hypotheses about adversary intent and tactics. It is the responsibility of the human operators to interpret this data and make the important decisions, both in terms of what probing actions to perform and how those actions achieve the objectives of the mission. To that extend, the human-machine interface (HMI) needs to display the results from the TA1 and TA2 technologies in an intuitive way so that the user can understand the tradeoffs between the different options, and make informed decisions based on levels of confidence of the results, the uncertainty inherent in the estimates, and the context of the situation. In addition, the HMI needs to engender trust and confidence in the results. The HMI must facilitate interactive knowledge development and provide mechanisms to convey complex algorithmic information in ways that is easily interpretable and expedient to support real-time human-machine decision making.

Extract features from multi-modality data and provide a unified data repository for downstream processing by the TA1 and TA2 performers. TA3 performers will develop technologies to automatically extract features about entities (actors, groups, organizations, states, etc.), relationships (association types, dependencies, causal factors, etc.), events (types, timings, locations, actors, etc.), and dynamics (rates, statistics, states, etc.) in an operational environment of concern. DARPA is interested in technologies that can extract information across the PMESII spectrum of activities as listed in the table below.

Table 1: COMPASS will process data from sources across the full PMESII spectrum

Source type Data description Intelligence value News Keywords, topics, actors and events (phase 1)

Articles in mainstream publications (phase 2) Military, Political, Economic, Transit / Traffic

Provides real time (and historical) location for vehicles, taxis, subway, public transportation, etc.

Infrastructure

Culture Ethnic groups, diversity, religion, etc. Social

Census Ethnic data parsed by geographic region Social EMS Emergency management status (shelter info, hospital status, etc.,) Infrastructure, Non-state actors

Social Media

Keywords, topics, actors and events (phase 1) Microblogs, Media, forums, etc. (phase 2)

Social, Economic, Political, Information

Utilities Critical Infrastructure maps and models Infrastructure Economy Income, opportunity zones, unemployment Economic, Political Police reports

Crime, riots, protests, etc. Social, Non-state actors

Cyber Reports of services disruptions and cyber-attacks Information, infrastructure

TA3 proposals should clearly describe the types of information products (e.g., formal text, informal text, databases, mixed text/numeric) they will support and any anticipated progression over the course of the program in types of products that can be processed. TA3 performers are expected to leverage state of the art technologies to extract this information from the sources in Table 1. All textual data will be in English. TA3 will also need to map the features extracted into an ontology that supports reasoning by other components. TA3 proposals should include in their technical approach a conceptual description of an interface by which these products can be communicated to other components, identifying the standards to be employed and any extensions or new constructs unique to the proposed approach. It is the intent of the program that a common set of interfaces will be developed early in Phase 1 to which all TA1 and TA2 performers will conform.

Deliver a software framework that accommodates integration of diverse applications. As the system integrator, the TA3 performer is responsible for providing the information management and processing framework that will unite TA1 and TA2 decision aids into an integrated platform.

This includes providing scalable storage and retrieval, scalable distributed processing, and other shared services within a common architecture. The TA3 performer will develop the system architecture, define interfaces between the major components of the system, and develop information exchange standards that will enable components developed by multiple performers to be integrated into a common framework. They will maintain and provide both documentation of the system interfaces and implementations in the form of a software development kit (SDK). The TA3 performer will plan and execute a series of collaborative experiments that bring operational users and technology developers together to explore and refine concepts for end-user employment of the COMPASS capabilities. The TA3 performer will lead the integration activities to prepare the system for use in two performance evaluations, and one operational assessment, and will support the Government operational evaluation partners in conducting the evaluations.

Provide the processing facility for both unclassified and classified development and experimentation: The TA3 performer will provide a processing facility to be used by TA1 and

TA2 performers for independent development and testing, and to facilitate distribution and exchange of data sets and software prototypes. This processing facility may be physically hosted by the TA3 performer or it may use commercial cloud hosting services that can be certified for DoD use. The TA3 processing facility must be remotely accessible by all performers. The TA3 performer will provide a set of common services to enable and facilitate independent development and testing within the common architecture.

The COMPASS program will conduct testing with classified information up to the SECRET level.

TA3 proposals should include a plan to provide a SECRET testing facility by month 6 of Phase 2.

This facility will not require remote access. TA3 proposals should also include tasks for accreditation of the COMPASS software for installation onto both performer-hosted and Government processing facilities at the SECRET level. To minimize any schedule and cost risk due to problems in software accreditation, TA3 performers will provide advice and guidance to the Government and the other performers, from the beginning of Phase 1, regarding security-sensitive architectural issues such as the selection of open-source software libraries and supply chain risk management (SCRM) information pertaining to foreign developed hardware and software as applicable. TA3 performers will provide support to the Government evaluation partners on conducting the evaluations including delivery, installation, checkout, and orchestration of technical support.

e) Test and Evaluation Test and evaluation will be executed by a Federally Funded Research and Development Center (FFRDC), a University Affiliated Research Center (UARC), or similar organizations and is not being solicited under this BAA. DARPA provides the following information in an effort to help proposers gain a better understanding of the evaluation environment.

COMPASS technologies will demonstrate that probing provides intelligence value above and beyond traditional passive collection. Testing such a capability, however, is not feasible in a real setting, in particular a setting where the environment is under destabilizing effects from gray zone actors. To address these challenges, DARPA will perform testing in two dimensions. The first dimension will evaluate the complexity and scale of the technologies via a well-controlled framework that allows rigorous scientific experimentation and assessment. The T&E team will conduct the complexity / scale testing via a virtual-constructive simulation for an urban location and population in both phases of the program. The second dimension is realism. The purpose of realism testing is to assess the ability of the system to detect activities against real-world clutter and environments. Although limited in scope, the T&E team will conduct realism testing via a live surrogate experiment at the end of the program.

T&E activities will address the following topics:

Complexity / Scale Testing: The T&E team will evaluate TA1 and TA2 technologies in an environment that simulates an urban location and population. The T&E thrust is responsible for developing this constructive simulation, called ‘gray zone test range’, and for building the experimentation framework required to assess the capabilities and progress of the program. The test range will employ multi-resolution models to address situations with tactical short-time horizon (days to weeks) interactions to broad strategic outcomes over the course of years. These models capture characteristics of services in urban environments, such as critical infrastructure, transportation, etc., as well as crime, population grievances, governance, economy, etc. A key emphasis of these models is to assess the impact that disruptions in those urban domains have on the population, and how the adversary “weaponizes” those disruptions to advance its subversive intents. Strong emphasis will be placed on the interconnection of models that contribute to complexity and ambiguity, such as critical transitions that lead to instabilities. The test range should represent the breadth of gray zone activities such as cyber operations, information operations, political pressure, economic extortion, security force assistance, corruption, election interference, discord and confusion, etc. It must also generate the many possible futures inherent in complex, uncertain gray zone conflict scenarios.

The T&E team will conduct a sequence of technology assessments over the course of the program to provide the government with an independent assessment of the suitability and maturity of these technologies for the objectives of the program. Prior to the program kickoff, the government team will procure the data, identify the models for the virtual environment and provide all this material to the performers as GFE at program start. During program execution, development will occur at a tempo of six-month spirals by staggering development of both the ‘test range’ and the COMPASS technologies. At the end of each spiral, the government will conduct an integration event to ensure that the interfaces converge and that the architecture closes.

Technology providers and the T&E team should have a common understanding of how the virtual environment will behave in response to external stimuli, including both the subversive actions of the gray zone actor as well as the probing actions of blue forces. In an environment driven strictly by physical processes, such as critical infrastructure and transportation events, the impact of external shocks are well understood and can be expressed by mathematical equations that have been tested and refined across different situations. However, in environments that include social activities, such as influence campaigns and spreading corruption, the behavior is not well understood, and there are no globally accepted models that capture the dynamics of the system.

What does exist, however, is a plethora of models with theories of how a complex system ‘ought’ to behave under specific conditions, such as the DARPA COMPOEX and ICEWS programs, the ONR Human, Social, Cultural, Behavioral program, and various commercial models for economics, politics, social characteristics, and others. DARPA is interested in leveraging this body of work. Technical approaches that extend existing models to accommodate probing actions, or approaches that instantiate existing theories as computational models, are within the scope of the program. The T&E team and the performers are expected to identify the socio-physical models they intend to use for their technical approach and describe the first principles that underlie the theory behind those models. At program kickoff, DARPA will hold a coordination event in which the T&E team will evaluate the models proposed by the technology providers, and through negotiations determine which ones to include in the test range.

Realism Testing: At the end of the Phase 2, the T&E team will augment the test range with a realism experiment that involves a live, surrogate event across a range of options in a military range setting. The realism experiment will allow the Government to assess the value of the technologies produced. The venue and type of the realism experiment will be defined after program start.

In addition to leading the Complexity/Scale and Realism evaluation efforts, T&E is also responsible for defining program level metrics that measure program success. An initial list of metrics is provided in section I.B. The T&E team will collaborate with COMPASS performers and other stakeholders to refine and extend metrics over time.

The T&E team is also responsible for the following tasks:

1. Assemble data sets for technology development and assessment

2. Develop a methodology that take into account the human-on-the-loop nature of the COMPASS process and will consider the relationship between the level of human involvement required and the suitability of the resulting intelligence products

3. Assist the local partner providing the T&E venue with the coordination and execution of the realism experiment.

4. Schedule/Milestones Proposers may submit to one or more TAs, but separate proposals must be submitted for each TA.

Each proposal must consider the full program timeline and should provide a technical and programmatic strategy that conforms to the entire program schedule. Proposals must clearly present an aggressive plan to fully address all program goals, metrics, milestones and deliverables as they pertain to each topic of concern within each TA. A target start date of September 2018 may be assumed for planning purposes.

The COMPASS program is planned as a two phase, 30-month effort with Phase 1 lasting 12 months and Phase 2 lasting 18 months. This BAA solicits proposals for both phases, with Phase 2 as a priced option. Phase 1 will end with an experiment that will measure the performance of the technologies developed.

Figure 4: COMPASS is a technology demonstration program, with integration events designed to demonstrate incremental progress of capabilities. Performance of the program is evaluated with three experiments.

Proposers should assume the availability of an unclassified data set at program start, with additional data sets provided at key program milestones as specified by the Program Event table (see Table 2). The developmental data sets provided by the Government are intended to provide a common baseline against which the performance of multiple approaches can be assessed.

However, proposers are encouraged to identify and use additional data sets. To facilitate TA1 and TA2 development, TA3 performer will be expected to periodically provide data output sets that are derived from Government-provided input sets, as described in the Schedule and Milestones section below. For output sets provided early in Phase 1, manual post-processing to minimize errors is acceptable.

a) Phase 1 In Phase 1, TA1 and TA2 performers will develop basic algorithms that infer campaigns of an inferior adversary with relatively modest goals. The performers will develop simple probing actions based on scripted scenarios. They will determine gaps in their knowledge of the adversary and come up with actions that solicit responses to fill those gaps. Moreover, these actions will account for different types of physical disturbances and be robust enough to maintain stability in the operating environment and avoid undesirable effects (such as escalation). The system integrator performer (TA3) will be responsible for the development of simple visualization tools focusing on physical characteristics of the environment; for developing the information management system that ingests, manages, and provisions the system data; and for building the software framework that integrates the probing technologies into the experimentation testbed.

There will be minimal emphasis upon integrated capability demonstration, but there will be focus upon identifying interfaces and data standards, as well as identifying gaps between the needs of users and those being developed in technical areas 1 and 2 (gap identification will be conducted through working groups, with participation from all performers).