HR001118S0010.pdf

Broad Agency Announcement Configuration Security (ConSec)

HR001118S0010

December 11, 2017

Defense Advanced Research Projects Agency Information Innovation Office 675 North Randolph Street Arlington, VA 22203-2114

HR001118S0010 CONFIGURATION SECURITY 2

Table of Contents

I. Funding Opportunity Description A. Introduction B. Program Description and Structure C. Technical Areas D. Evaluation and Demonstration E. Schedule and Milestones F. Deliverables to DARPA G. Government-furnished Property/Equipment/Information H. Intellectual Property

II. Award Information A. Awards B. Fundamental Research C. Disclosure of Information and Compliance with Safeguarding Covered Defense Information Controls

III. Eligibility Information A. Eligible Applicants B. Organizational Conflicts of Interest C. Cost Sharing/Matching D. Other Eligibility Requirements

IV. Application and Submission Information A. Address to Request Application Package B. Content and Form of Application Submission C. Submission Dates and Times D. Funding Restrictions E. Other Submission Requirements

V. Application Review Information A. Evaluation Criteria B. Review and Selection Process

VI. Award Administration Information A. Selection Notices B. Administrative and National Policy Requirements C. Reporting

VII. Agency Contacts VIII. Other Information

A. Frequently Asked Questions (FAQs) B. Proposers Day C. Submission Checklist D. Associate Contractor Agreement Clause (ACA)

HR001118S0010 CONFIGURATION SECURITY 3

PART I: OVERVIEW INFORMATION

Federal Agency Name: Defense Advanced Research Projects Agency (DARPA), Information Innovation Office (I2O)

Funding Opportunity Title: Configuration Security (ConSec)

Announcement Type: Initial Announcement

Funding Opportunity Number: HR001118S0010

Catalog of Federal Domestic Assistance Numbers (CFDA): 12.910 Research and Technology Development

Dates o Proposers Day: November 17, 2017 o Posting Date: December 11, 2017 o Abstract Due Date: December 22, 2017, 12:00 noon (ET) o Proposal Due Date: February 8, 2018, 12:00 noon (ET) o BAA Closing Date: February 8, 2018, 12:00 noon (ET)

Anticipated Individual Awards: The Government anticipates one or more awards for Technical Area (TA) 1, multiple awards in TA2, and single awards for TA3 and TA4.

Total Funding Available for Award: $45 million.

Types of Instruments that May be Awarded: Procurement contracts or cooperative agreements

Agency Contacts o Technical POC: Mr. Jacob Torrey, Program Manager, DARPA/I2O o BAA Email: ConSec@darpa.mil o BAA Mailing Address:

DARPA/I2O

ATTN: HR001118S0010

675 North Randolph Street Arlington, VA 22203-2114 o I2O Solicitation Website: http://www.darpa.mil/work-with-us/opportunities mailto:ConSec@darpa.mil http://www.darpa.mil/work-with-us/opportunities

HR001118S0010 CONFIGURATION SECURITY 4

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

DARPA is soliciting innovative research proposals to develop technologies for automatically analyzing and improving the configuration of complex composed systems to reduce the attack surface while assuring expected behavior. Proposed research should investigate innovative approaches that enable revolutionary advances in science, devices, or systems. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice.

This Broad Agency Announcement (BAA) is being issued, and any resultant selection will be made, using procedures under Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016.

Any negotiations and/or awards will use procedures under FAR 15.4 (or 2 CFR § 200.203 for cooperative agreements). Proposals received as a result of this BAA shall be evaluated in accordance with evaluation criteria specified herein through a scientific review process.

DARPA BAAs are posted on the Federal Business Opportunities (FBO) website (https://www.fbo.gov/) and, when applicable, the Grants.gov website (http://www.grants.gov/).

The following information is for those wishing to respond to this BAA.

A. Introduction

The growth of the internet-of-things (IoT) and network-connected composed systems (e.g., aircraft, critical-infrastructure, etc.) has led to unprecedented technical diversity in deployed systems. From consumer IoT devices developed with minimal built-in security, which are often co-opted by malware to launch large distributed denial of service (DDoS) attacks on internet infrastructure, to remote attacks on Industrial Control System (ICS) devices, these newly-connected, composed systems provide a vast attack surface. While the diversity of functionality and the scope of what can now be connected, monitored, and controlled over the network has increased dramatically, economies of scale have decreased platform diversity. Inexpensive commodity off-the-shelf (COTS) devices have largely replaced single-purpose, custom devices.

For example, the central processing unit (CPU) market has consolidated on ARM, x86 and stream processors.

These economies of scale have not only influenced the consumer markets, but also industrial and military platforms, where special-purpose, custom-built, components have been largely replaced with cheaper commodity components programmed to provide similar functionality. This shift has opened new attack surfaces, as software and configuration settings now govern behaviors that were physically impossible in special purpose hardware.

Vendors of these commodity components have a strong incentive to make their products flexible and general-purpose in order to make them applicable to a broad range of possible deployment configurations. However, this flexibility puts the burden of security on the system owner, who must configure these components in such a way to reduce attack surface derived from unwanted functionality. Proper configuration aligns general-purpose COTS devices with their intended use-cases (Figure 1). The ConSec program will develop an automated capability to configure the components of a target system to provide expected functionality while minimizing attack https://www.fbo.gov/ http://www.grants.gov/

HR001118S0010 CONFIGURATION SECURITY 5

surfaces.

Figure 1: Configuration used to change functionality

Humans are part of cyber-physical systems in that they provide control inputs. Human-readable standard operating procedures (SOPs) describe sequences of actions that operators should take in response to an event or notification from the cyber-physical system. For example, pilots follow checklists that detail the steps required to perform a certain action or respond to an equipment error condition. The ConSec capability must be able to understand these interactions and generate them appropriately as a stand-in for human input, to ensure they are considered when analyzing the system’s security posture.

B. Program Description and Structure

The ConSec program will develop a system to automatically generate, deploy, and enforce configurations of components and subsystems for use in military platforms. These configurations should address system vulnerabilities and minimize attack surfaces while maintaining expected functionality and performance (Figure 2). By viewing each individual component’s configuration as elements of the composed system’s behavior and security, more secure configurations can be developed and deployed to enhance security without requiring new software development or large hardware changes.

Achieving these goals will require research breakthroughs in:

Deriving a functional specification for a component and analyzing how settings in its configuration space could impact its functionality, producing useful configuration semantic models without exhaustive exploration of the configuration space, and reasoning effectively with incomplete information.

Constructing models of intended functionality for the composed system with minimal human-in-the-loop time by understanding the operational context(s) of the composed system.

Ingesting standard operating procedures (e.g., pilot check-lists) that describe the operator’s interactions with composed systems and mapping them into functional models of system behavior, Characterizing attack surfaces stemming from poorly configured or composed components, and developing approaches to remedy those weaknesses via configurations.

Deploying secure configurations, monitoring them for changes during operation, and producing context-relevant responses in the event of an identified change.

Designing authoritative and auditable configuration repositories that provide strong integrity protections.

HR001118S0010 CONFIGURATION SECURITY 6

ConSec will consist of three phases (see schedule). Phase 1 will be 15 months in duration, and will emphasize initial development of the tools and techniques needed to ingest operational context information, configuration, human SOPs, and to model the target system’s intended behavior. The scale of the target system in Phase 1 will be on the order of a home network with building automation and IoT components, or a commercial vehicle. Phase 2 will be 15 months in duration, and will focus on securing systems via new configurations, and providing human-understandable evidence for why those configurations are optimal. The scale of target system in Phase 2 will be on the order of a heavy industrial platform, or small SCADA/ICS deployment.

Phase 3 will be 12 months in duration, and will focus on augmenting secure configurations with run-time enforcement and system attestation, and working with identified transition partners. For Phase 3, two separate DoD systems will be used for testing and demonstration, each on the order of a naval platform subsystem or air operations center subsystem.

In Phase 1, there will be two integration/demonstration events and a final evaluation exercise at the end of the phase. Phases 2 and 3 will also have two demonstration events each to identify and correct any weaknesses, and provide ample time to address any shortcomings before mid-phase and final evaluation exercises.

C. Technical Areas

ConSec will be structured with four technical areas (TA):

TA1 - Understanding the Composed System TA2 - Generate Secure Configurations TA3 - Voice of the Offense TA4 - System Integrator and Evaluator

The Government anticipates one or more awards for TA1, multiple awards for TA2, and single award each for TA3 and TA4. Proposals shall address only one technical area. Proposers may submit multiple proposals for any or all four TAs.

TA2 performers may elect to build and integrate their tools with the rest of the ConSec system such that the performers themselves are not exposed to DoD transition systems. In this case, the research performed by the TA2 performer could be considered fundamental research. A TA2 performer that must have access to specific information regarding a DoD transition system will not be considered fundamental research and may require security clearances and secure facilities up to the TOP SECRET level. (See also II. Fundamental Research).

There are several points of potential collaboration among TAs, and the Government expects that all performers producing software will interact closely with the System Evaluator (TA4).

Proposers should read the descriptions of all TAs and the Evaluation and Demonstration section to ensure a full understanding of the program context, structure, and anticipated relationships required among performers. To facilitate the open exchange of information, performers will have an ACA clause included in their award. TA4 will lead the development of the ACA for the program. See Section VIII.D for more information regarding the ACA.

HR001118S0010 CONFIGURATION SECURITY 7

Figure 2: Overview of the ConSec capability

Relevant systems for secure configuration are composed of subsystems, which in turn are composed of electronic commercial devices (e.g., routers, programmable logic controllers) or components (e.g., hard disks). The following discussion will use the term component to refer to all such configurable constituent parts of relevant systems, and will refer to these systems as target systems.

TA1 - Understanding the Composed System

TA1 will produce a model of the required functionality for each operational context the target system must operate within, as well as per-component configuration parameters mapped to semantic functional specifications for use during configuration-time (Figure 3). An operational context is a specific mode of execution that the target system must support in order to successfully meet mission needs. TA1 must infer all relevant operational contexts from the documentation provided. For example, a naval vessel at sea may require different functionality than while at port, or while in dry-dock undergoing maintenance. TA2 systems will use this information to generate secure configurations for specific operational contexts of the target system. These resultant configuration sets will then be deployed by TA1 to the target system and monitored for deviation.

TA1 systems will infer the semantics of the target system and, based on this information, produce machine-readable specifications and models for TA2 systems to analyze. TA1 systems must be capable of analyzing the software/firmware of the component to be configured and producing a formal mapping from configuration settings to functionality. At a minimum, the TA1 system should support analysis of x86, ARM, and MIPS software/firmware. Strong proposals will discuss methods to infer high-level functionality from components based on other architectures or without binary access.

The Government will also provide TA1 performers with human-readable SOPs, user manuals, and other system documentation for the system under test. From this information, the TA1 system must output machine-readable functional requirements of the relevant operational

Figure 3: Nominal TA1 configuration-time subsystem

HR001118S0010 CONFIGURATION SECURITY 8

contexts for the system and a model of human operator interaction with the system’s control-flow assuming adherence to the SOPs. At a minimum, the TA1 system must ingest unstructured plain text input. Strong proposals will discuss methods to ingest information from graphics (e.g., network diagrams) and other design artifacts. ConSec’s goals preclude lengthy manual processes and clean-slate redesign, so TA1 proposals should focus on automation and reduction of human-in-the-loop effort. Where human attention is needed to refine the generated models, proposals should describe necessary human-computer interactions and how the proposed approach will support users who are not security or configuration experts.

TA1 systems must apply the semantic understanding of available configuration parameters they have derived to reduce the scale of the models they provide to TA2.

As part of the run-time subsystem, TA1 should deploy TA2-generated configuration sets to the target system, along with an enforcement mechanism that ensures these settings are auditable and resistant to tampering. TA1 proposals should describe approaches to configuration setting deployment and enforcement. Once the configuration set has been deployed for the selected operational context, a runtime monitoring capability that becomes part of the target system should enable rapid detection of changing configurations on system components. Such changes are either indicative of compromise, or transition from one operational context to another; in the latter case, TA1 should select a new configuration set for deployment with human authorization via the TA2 operator interface. Strong TA1 proposals will discuss how the generated configuration sets are stored and protected.

TA1 will be responsible for developing the technical data sharing standards for interactions with

TA2.

Strong TA1 proposals will, at a minimum, address the following challenges:

Semantic extraction from human-readable system documentation to produce a machine-readable model of how configuration parameters affect system behavior.

Reduction of the configuration space for each component to areas of greatest concern with respect to security and performance.

Discovering and modeling the relevant operational contexts of the target system from human-readable documentation, with minimal human-in-the-loop effort.

Binary and source analysis of the component firmware/software to automatically generate a specification of each component’s functionality based on its configuration.

Collaboratively creating a common data representation for communicating these developed models and specifications to the TA2 system that can normalize different vendor configuration conventions.

Developing an acquisition and deployment system to access, modify, and monitor the configuration parameters on all of the diverse devices of a large and complex system or platform.

TA2 - Generate Secure Configurations

During configuration time, TA2 systems (Figure 4) will analyze the functional model and formal representations of component documentation, SOPs, and the configuration-to-functionality mappings from TA1 to generate secure configuration sets for the overall system, one for each relevant operating context (e.g., deployed, undergoing maintenance). If human guidance is

HR001118S0010 CONFIGURATION SECURITY 9

needed to add constraints not modeled by TA1, a TA2-developed operator interface should interact with the human operators to better understand how the system should be configured.

TA2 configurations must preserve the functionality of the composed system (including non-functional requirements such as timing guarantees) while removing unneeded functionality and adjusting settings to eliminate attack surfaces and other inefficiencies. TA2 systems should generate supporting evidence to provide a rationale for their configuration settings. This body of evidence should enable transition partners to trust that the new configuration will allow the system to achieve its operational goals while substantially increasing its cyber security.

Figure 4: Nominal TA2 system

Compositional interactions between components will require thorough analysis to prevent the behavior of one component from impacting another in a negative manner.1 TA2 systems should therefore have the capability to analyze the system as a whole, so that they can identify and address such cross-component interactions. These developed configuration sets must then be delivered to TA1 for deployment (blue box in Figure 4). Each of the configuration sets should include a human-readable description of the operational context and how the transition between contexts should occur. These context notations, along with explanations of why suspect configuration changes should be examined will be communicated to the operator during runtime via the operator interface.

Successful TA2 proposals will, at a minimum, address the following challenges:

Performing compositional analysis of the TA1-provided system models and specifications to determine an optimal configuration set for the target in each operational context

Automatically generating human-readable evidence supporting the selected configuration set to facilitate transition

Communicating with the human operators during configuration and run time in order to refine models and explain run time actions

1 For example, if the maximum transmission unit (MTU) on a switch is set above the standard value of 1518, many modern devices will continue to operate as expected, but legacy devices may be unable to receive these oversized packets (jumbo frames) when the network is highly utilized.

HR001118S0010 CONFIGURATION SECURITY 10

TA3 - Voice of the Offense

TA3 will develop tools, techniques, and procedures to produce representative configuration-based vulnerabilities in complex composed systems. TA3 will operate in a “white-box” manner, with full access to all output from TA1 and TA2, as well as all human-readable documentation.

For each of the test target systems, TA3 will first review the standard configuration settings and modify them as needed to provide a sufficiently broad attack surface. This injection of configuration vulnerabilities will provide a baseline for evaluating TA1 and TA2 configuration sets. At the completion of an exercise, TA3 must restore the target system testbed to a known-good state for return to system owner. TA3 will also support the security of the ConSec software itself via architecture review and collaboration with TA1 and TA2 to prevent attackers using ConSec to weaken the security of the system.

Figure 5: Exploiting excess as-configured functionality

TA3 will operate under modified rules of engagement versus traditional penetration testing, in that all attacks on target systems must be accomplished solely through exploitation of vulnerabilities in the configuration sets produced by TAs 1 and 2 (Figure 5). Memory corruption, return-oriented programming attacks and other software-level exploitation are specifically excluded when testing target systems. The assessed performance of TA3 will depend on the level of automation for generating attack paths within the aforementioned constraints, and measured improvements in TA1 and TA2 systems, so collaboration with the other performers will be essential.

Successful TA3 proposals will, at a minimum, address the following challenges:

Develop tools, techniques, and procedures to exploit target systems solely via configuration- or composition-enabled vulnerabilities Assess potential to exploit the ConSec system as an enabler for attacking the system Produce initial configuration set with added vulnerabilities then evaluate security improvements after TA1 and TA2 have deployed a new configuration set Automation of TTPs developed in Phase 1 for increasing scalability of TA3 support for the larger target systems in Phases 2 and 3, minimizing human-in-the-loop effort

TA3 proposers should submit a base proposal assuming one performer in each of the first two technical areas (TAs 1-2), and two additional cost options, one to cover the incremental cost of an additional performer in TA1, and another for the incremental cost of an additional TA2 performer. All additional costs should span the entire ConSec program period of performance.

The Government will determine the total contract value to award during contract negotiations based on the selection of performers for awards and the nature of their proposed efforts.

HR001118S0010 CONFIGURATION SECURITY 11

TA4 - System Integrator and Evaluator

The TA4 performer will be responsible for evaluating the performance of TA1 and TA2 systems against ConSec metrics and integrating TA1 and TA2 systems into configuration-time and run-time subsystems. ConSec will automate a currently manual process, therefore regular evaluation of the TA1 and TA2 systems will support the transition in addition to the specific TA2-developed supporting evidence. TA4 proposals should discuss additional, objective metrics for determining the security improvement of the overall ConSec system, as well as methods to ensure that essential functionality has not been removed or changed through the deployment of an incorrect configuration set.

TA4 must propose a simple simulated/emulated testbed that can initially be provided to ConSec performers 3 months after program start. This testbed can be augmented over the duration of the effort to facilitate automated regression testing and evaluation. TA4 should expect containerized software from TA1 and TA2 that interface via the agreed-upon common data format that when composed create both the configuration-time and run-time ConSec subsystems. TA4 will manage the integration process with the assumption that TA1 and TA2 software is sufficiently mature to preclude lengthy bug-finding on the part of TA4.

The TA4 performer will be responsible for coordinating demonstrations and exercise evaluations on DARPA-selected target systems, as well as integrating the development of TA1 and TA2 into configuration-time and run-time subsystems. Over the three phases of the ConSec program, the TA4 performer will work with the Government team and testbed system owners to provide efficient evaluations and measurements of ConSec effectiveness, functionality, and user experience. The TA4 performer must also validate that TA3 has restored testbed systems to a verified, known state at the end of each exercise. See Section I.D below for more information on ConSec exercises.

The TA4 integrator will be responsible for coordinating the development of detailed interface specifications and overall system design with TA1 and TA2. Starting in the middle of Phase 1, TA4 will integrate the components from TA1 and TA2 into configuration-time and run-time subsystems. These subsystems should be tested in an automated fashion after each two month’s code delivery on the aforementioned simple surrogate system of TA4’s devising to prevent regressions. A regression and results report should be provided to the performers and the DARPA team after each code delivery.

TA4 will lead the development of the ACA for the program. See VIII.D for more information regarding the ACA.

Successful TA4 proposals will, at a minimum, address the following challenges:

Facilitation of common data formats between TA1 and TA2 that can abstract vendor-specific parameter names and semantics Coordination of exercises on limited physical target systems and how to quickly verify that the testbed is in a known good state Development of novel metrics and a system analysis framework to evaluate the progress and viability of the ConSec system Creation of a simple test platform to allow for automated regression testing of TA1 and

TA2 deliverables

HR001118S0010 CONFIGURATION SECURITY 12

Integration of TA1 and TA2 tools into a cohesive workflow for future transition

TA4 proposers should submit a base proposal assuming one performer in each of the first two technical areas (TAs 1-2), and two additional cost options. The additional costs should cover: the incremental cost of an additional performer in TA1 and the incremental cost of an additional performer for TA2. The Government will determine the total contract value to award during contract negotiations based on the selection of performers for awards and the nature of their proposed efforts. TA4 proposals should also include an option scoped for 1-2 FTEs to begin performance at the end of the 42 month program period of performance and continue for an additional 18 months to assist in supporting ConSec transition.

D. Evaluation and Demonstration

The ConSec program System Evaluator (TA4) will assist the Government team in the development of evaluations to provide feedback to TA1 and TA2 performers. These evaluations will include demonstrations of the ConSec systems on a simulated target system and evaluation exercises on high-fidelity composed systems to characterize the capabilities that TA1 and TA2 performers produce.

DARPA will assess individual performer efforts in terms of the viability of their technical approaches, the trend in the performance of their systems over time, and their overall progress toward ConSec program objectives.

TA1 Phase 1 Phase 2 Phase 3 Model Fidelity 80% of static space 80% of static space 90% of static space Documentation ingest 10x manual

60% accuracy 10x manual 70% accuracy

10x manual 80% accuracy

Deployment time 1.5x faster than manual 5x faster than manual 30x faster than manual TA2 Phase 1 Phase 2 Phase 3

Configuration-space coverage

60% 75% 90%

Vulnerability reduction 85% 85% 85% Formalization of supporting evidence

10% 40% 80%

Figure 6: Envisioned metric progression

1. Evaluation DARPA expects all performers producing software to follow an agile software development process. Initial code drops should consist of largely stubbed-out, end-to-end systems, with capabilities added and refined incrementally over the period of performance. Code drops (delivered every two months) shall include all source code, build scripts, test harnesses, development environments, unit tests, and system tests that TA4 can readily use (e.g., a Vagrant or Docker container). See VIII.D for more information regarding the ACA.

TA1:

HR001118S0010 CONFIGURATION SECURITY 13

Figure 6 shows three metrics for TA1 system performance that measure ability to reduce human-in-the-loop effort while building a tractable, accurate model for TA2 to analyze.

The model fidelity metric measures the amount of the state-space that analysis of the component binaries captures. This state-space is characterized as either static (recoverable through purely static analysis) or dynamic (runtime state-space only available through dynamic analysis). Initial evaluation will be limited to static state-space as it is possible to estimate the static state-space analyzed by a tool or technique, whereas total dynamic state is more difficult to quantify. Documentation ingest measures the time required and resulting accuracy of TA1 machine reading relative to manual encoding into a domain-specific language of the human-readable documentation, SOPs and associated manuals. The deployment time metric sets goals for the time required to deploy these configuration sets across a diverse multi-vendor system.

TA2:

The three system metrics TA2 (see Figure 6) will measure are (a) the ability to explore the configuration space in terms of the optimality of the configuration settings, (b) how much attack surface has been trimmed from the default or existing configuration, and (c) the extent to which the body of evidence provides logical support for the new configuration settings. Strong TA2 proposals will discuss additional metrics that measure the security improvement with the newly devised configuration sets.

All TA1 and TA2 proposals must describe a set of metrics specific to the proposed approach.

In the first weeks of the program, each performer will collaborate with TA4 to produce a document defining the metrics for measuring their system’s performance in addition to those in Figure 6.

The System Evaluator (TA4) will develop and conduct largely automated testing on a two month basis to verify that each system builds and executes its tests properly. Each performer developing software will receive testing reports to assist their development efforts. Over the course of Phase 1, TA4 will build out their own test cases for each system, to augment performer-provided tests.

2. Demonstrations and Exercises Demonstration events will start at six months, to provide feedback to guide research and development efforts. Participation in six demonstrations (smaller, ConSec-internal event) and five exercises (larger events with more Government engagement) that increase in scale and realism over the course of the program will be the primary focus of ConSec evaluation. TA4 will lead the development of the testing scenarios and work with TA3 in order to devise the initial configuration and operational contexts of the target system.

Each demonstration will provide the Government team a chance to see the ConSec system operating against a target system that may be partially or wholly simulated. These target systems will be known by all performers and accessible for testing and development purposes prior to the demonstration. Exercises will operate on much higher fidelity systems where a limited test window approximate the application of ConSec to a new transition system, with less information on the exercise target available to performers a priori.

HR001118S0010 CONFIGURATION SECURITY 14

Initial demonstrations may be paper-based. In these events, the focus will gradually shift to the utility of the ConSec subsystems in the hands of operators. There will be eleven demonstrations and exercises over the life of the program. The first three exercises will be limited to ConSec performers, and their objective will be to familiarize each team with the target system. DARPA will arrange to have Government subject matter experts (SMEs) participate in each of these exercises to help performers understand the domain (e.g., industrial control system, maritime vessel subsystem, etc.) in sufficient detail. These SMEs will execute non-disclosure agreements (NDAs) with ConSec performers.

For costing purposes, proposers should assume that all demonstrations will take place in the Washington, D.C. metro area, and will run for two days in conjunction with PI meetings.

Assume that PI meetings will require 1.5 days in addition to the demonstration, and will be held in the Washington, D.C. area.

For costing purposes, proposers should assume that the first exercise will take place in the Washington, D.C. metro area, and will run for two days. The majority of each team’s personnel should be present at these three demonstrations/exercises in order to become familiar with the domain. Assume for the other exercises that the location will alternate between San Diego, CA, Denver, CO, and the Washington, D.C. metro area. Each of the remaining exercises will require at least three technical team members to be onsite at the event location for one week.

Close collaboration is expected on this effort. Proposers of TA1 and TA2 will have to work closely to coordinate common data representations for component specifications and configurations, human processes, and functional requirement models. A more detailed table is provided below to call attention to a subset of the expected touch-points between performers.

An understanding of the metrics used to evaluate every TA will help inform the responsibilities and dependencies between performers.

TA Required Collaboration

TA1 With TA2: Collaboratively devise a representation of the incomplete functional specifications mapped to configuration parameters. Agree upon a method, or domain-specific language for normalizing the component parameters and their semantics. This method should be rich enough to express the semantics of the human operator behavior and the operational context(s) in which the system can exist that TA2 can reason over it. Coordination of how the TA2 configuration sets should be deployed and enforced based on changing operational contexts will be needed.

With TA3: Plan how to coordinate sharing limited physical testing systems. TA1 is responsible for providing TA3 with the same model and behavior information developed with TA2 so that TA3 can develop attack TTPs.

With TA4: Coordinate common data formats for representing the target system, functional requirements, and model with which TA4 can calculate metrics-based analyses. Also coordinate with TA4 on interfacing TA1 tools into the unified workflow and the plan for sharing the limited physical testing systems.

HR001118S0010 CONFIGURATION SECURITY 15

TA2 With TA3: Format information for how TA2 identifies and represents the excess attack surfaces. Unneeded and/or duplicate functionality must be communicated to TA3. TA3 will need this information to improve TA3 tooling for attack-path generation.

With TA4: Coordinate common data formats for representing the configuration space of the target system to enable TA4 to evaluate coverage analysis. Coordinate with TA4 on how the TA2 tools should be interfaced into the unified workflow.

TA1 &

TA2

Co-integration requirements with TA4: Coordinate interfaces for integration by TA4 into single workflow for both configuration-time and run-time ConSec subsystems. Ensure TA1 and TA2 software is containerized and regression-tested prior to delivery to minimize TA4 effort for integration.

TA3 With TA4: Coordinate sharing the limited physical testing systems to ensure both TA3 is able to measure the security improvement and TA4 is able to evaluate the metrics for the TA1 and TA2 performers when applied to the testing system. Support TA4 in identifying behavioral edge-cases that may be discovered by automated tooling that does not impact security, but may improve test coverage.

E. Schedule and Milestones

DARPA anticipates a July 2018 start date for the ConSec program. The program will run for 42 months, and will comprise three phases. Phases 1 and 2 will be 15 months each, and Phase 3 will be 12 months, as shown in Figure 7. There will be biannual PI meetings held in conjunction with demonstrations to review technical progress and provide a venue for face-to-face collaboration.

TA4 system evaluations will be conducted one month prior to PI meetings in order for results to be available for discussion.

Figure 7: ConSec schedule

Phase 1 (Scaffolding) will focus on developing end-to-end systems and core technical capabilities. There will be a demonstration and evaluation of initial TA1 and TA2 capabilities at month 6, an initial demonstration at month 12 to familiarize performers with the issues they are likely to encounter, and a more involved demonstration/exercise of Phase 1 systems at month 15.

For scope, proposers can assume the target system in Phase 1 will be on the scale of a small cyber-physical environment such as a home-automation network, commercial vehicle, or

HR001118S0010 CONFIGURATION SECURITY 16

physical security control network in a high-risk environment.

Phase 2 (Initial Deployment) will extend the Phase 1 capability to produce systems and tools suitable for use by a government operator with a strong understanding of the system in collaboration with the ConSec performer team. There will be two exercises in Phase 2, the latter involving anticipated transition system users. Proposers can assume the Phase 2 target will be on the scale of a freight locomotive, a power sub-station, or an assembly-line on a factory floor.

Phase 3 (Continuous Improvement) will focus on producing a scalable, efficient, and deployable capability. There will be three exercises conducted at considerably larger scale than the Phase 2 exercises. In order to demonstrate that the ConSec-developed system is flexible to be transitioned to a variety of DoD systems, Phase 3 evaluations will be performed on two transition DoD systems. These systems will be an order-of-magnitude more complex than that of Phase 2, and can be assumed to be on the order of a naval maritime platform subsystem, autonomous convoy of ground vehicles, or an air operations center subsystem.

F. Deliverables to DARPA

All FAR-based contract performers will be required to provide, at a minimum, the following deliverables:

Any technical papers derived from work funded by ConSec;

Commented source code, any other necessary data and documentation (including at minimum user manuals and a detailed software design document) for all software developed under this program;

For all performers developing software, code drops will be provided to the System Evaluator every two months, to include all source code, build scripts, test harnesses, development environments, unit tests and system tests;

For all performers developing software, no later than week 9 after the start of each phase, a document defining metrics for testing and evaluation and discussing a concept of operations for conducting evaluations of any software that requires user interaction, to be produced in collaboration with the System Evaluator;

Annotated slide presentations must be submitted within one month after the program kickoff meeting and after each program event (program reviews, PI meetings, and technical interchange meetings);

Monthly technical status reports detailing progress made, tasks accomplished, major risks, planned activities, trip summaries, changes to key personnel, and any potential issues or problem areas that require the attention of the Government Team must be provided within 10 days of the end of each calendar month;

Monthly financial status reports must be provided within 10 days of the end of each calendar month;

A final report for each program phase that concisely summarizes the effort conducted, technical achievements, and remaining technical challenges will be due 30 days after the end of each phase; and

A final report at the end of the overall period of performance that summarizes the project.

In addition, the following deliverables are required for particular technical areas:

TA2: Documentation for generated supporting evidence that the configuration sets reduce

HR001118S0010 CONFIGURATION SECURITY 17

vulnerabilities while providing the necessary functionality.

TA3: System under test validation report for each of the target systems (one each for Phases 1 and 2, two for Phase 3).

TA4: Starting no later than 6 months after kickoff, the System Evaluator must provide to the Government all testing/evaluations metrics for evaluating TA1 and TA2 performers; quarterly status reports on the state of each TA1 and TA2 performer’s technical progress (these reports must include quantitative results, including all data collected in a digital format suitable for further analysis, and metric results based on these measurements). The System Evaluator must also establish a private program Wiki to facilitate collaboration and information sharing amongst ConSec performers.

G. Government-furnished Property/Equipment/Information

The Government intends to furnish multiple sources of data; however, performers should propose sources of data relevant to their particular approaches. In some cases, it may be easier for the Government to access such data. The Government also intends to furnish access to the target composed systems or high-fidelity simulators for testing and evaluation.

H. Intellectual Property

A key goal of the program is to establish an open, standards-based, plug-and-play architecture that allows for interoperability and integration across diverse target systems. This includes the ability to easily add, remove, substitute, and modify software and hardware components. This will facilitate rapid innovation by providing a base for future users or developers of program technologies and deliverables. Therefore, it is desired that all software (including source code), associated documentation, hardware designs and documentation, and technical data generated by the program be provided as deliverables to the Government with a minimum of Government Purpose Rights (GPR), as lesser rights may adversely impact the lifecycle costs of affected items, components, or processes. See Section VI.B.1 for more details on intellectual property.

HR001118S0010 CONFIGURATION SECURITY 18

II. Award Information

A. Awards

The Government anticipates one or more awards for TA1, multiple awards for TA2, and single awards for TA3 and TA4. The level of funding for individual awards made under this solicitation has not been predetermined and will depend on the quality of the proposals received and the availability of funds. Awards will be made to proposers whose proposals are determined to be the most advantageous and provide the best value to the Government, all factors considered, including the potential contributions of the proposed work, overall funding strategy, and availability of funding. See Section V for further information.

The Government reserves the right to:

select for negotiation all, some, one, or none of the proposals received in response to this solicitation;

make awards without discussions with proposers;

conduct discussions with proposers if it is later determined to be necessary;

segregate portions of resulting awards into pre-priced options;

accept proposals in their entirety or to select only portions of proposals for award;

fund proposals in increments and/or with options for continued work at the end of one or more phases;

request additional documentation once the award instrument has been determined (e.g., representations and certifications); and remove proposers from award consideration should the parties fail to reach agreement on award terms within a reasonable time or the proposer fails to provide requested additional information in a timely manner.

Proposals selected for award negotiation may result in a procurement contract or cooperative agreement, depending upon the nature of the work proposed, the required degree of interaction between parties, and other factors.

In all cases, the Government contracting officer shall have sole discretion to select award instrument type, regardless of instrument type proposed, and to negotiate all instrument terms and conditions with selectees. DARPA will apply publication or other restrictions, as necessary, if it determines that the research resulting from the proposed effort will present a high likelihood of disclosing performance characteristics of military systems or manufacturing technologies that are unique and critical to defense. Any award resulting from such a determination will include a requirement for DARPA permission before publishing any information or results on the program. For more information on publication restrictions, see the section below on Fundamental Research.

B. Fundamental Research

It is DoD policy that the publication of products of fundamental research will remain unrestricted to the maximum extent possible. National Security Decision Directive (NSDD) 189 defines fundamental research as follows:

‘Fundamental research’ means basic and applied research in science and engineering, the results of which ordinarily are published and shared broadly within the scientific community, as distinguished from proprietary research and from industrial development, HR001118S0010 CONFIGURATION SECURITY 19 design, production, and product utilization, the results of which ordinarily are restricted for proprietary or national security reasons.

As of the date of publication of this BAA, the Government expects that program goals as described herein may be met by proposers intending to perform fundamental research and proposers not intending to perform fundamental research or the proposed research may present a high likelihood of disclosing performance characteristics of military systems or manufacturing technologies that are unique and critical to defense. Based on the nature of the performer and the nature of the work, the Government anticipates that some awards will include restrictions on the resultant research that will require the awardee to seek DARPA permission before publishing any information or results relative to the program.

Proposers should indicate in their proposal whether they believe the scope of the research included in their proposal is fundamental or not. While proposers should clearly explain the intended results of their research, the Government shall have sole discretion to select award instrument type and to negotiate all instrument terms and conditions with selectees. Appropriate clauses will be included in resultant awards for non-fundamental research to prescribe publication requirements and other restrictions, as appropriate. This clause can be found at http://www.darpa.mil/work-with-us/additional-baa.

For certain research projects, it may be possible that although the research being performed by the awardee is restricted research, a subawardee may be conducting fundamental research. In those cases, it is the awardee’s responsibility to explain in their proposal why its subawardee’s effort is fundamental research

C. Disclosure of Information and Compliance with Safeguarding Covered Defense Information Controls

The following provisions and clause apply to all solicitations and contracts; however, the definition of “controlled technical information” clearly exempts work considered fundamental research and therefore, even though included in the contract, will not apply if the work is fundamental research.

DFARS 252.204-7000, “Disclosure of Information” DFARS 252.204-7008, “Compliance with Safeguarding Covered Defense Information Controls” DFARS 252.204-7012, “Safeguarding Covered Defense Information and Cyber Incident Reporting”

The full text of the above solicitation provision and contract clauses can be found at http://www.darpa.mil/work-with-us/additional-baa#NPRPAC.

Compliance with the above requirements includes the mandate for proposers to implement the security requirements specified by National Institute of Standards and Technology (NIST) Special Publication (SP) 800-171, “Protecting Controlled Unclassified Information in Nonfederal Information Systems and Organizations” (see https://doi.org/10.6028/NIST.SP.800-171r1) that are in effect at the time the BAA is issued, or as authorized by the Contracting Officer, not later than December 31, 2017.

http://www.darpa.mil/work-with-us/additional-baa http://www.darpa.mil/work-with-us/additional-baa#NPRPAC https://doi.org/10.6028/NIST.SP.800-171r1

HR001118S0010 CONFIGURATION SECURITY 20

For awards where the work is considered fundamental research, the contractor will not have to implement the aforementioned requirements and safeguards; however, should the nature of the work change during performance of the award, work not considered fundamental research will be subject to these requirements.

HR001118S0010 CONFIGURATION SECURITY 21

III. Eligibility Information

A. Eligible Applicants

DARPA welcomes engagement from all responsible sources capable of satisfying the Government's needs, including academia (colleges and universities); businesses (large, small, small disadvantaged, etc.); other organizations (including non-profit); entities (foreign, domestic, and government); FFRDCs; minority institutions; and others.

DARPA welcomes engagement from non-traditional sources in addition to current DARPA performers.

1. Federally Funded Research and Development Centers (FFRDCs) and Government Entities

a. FFRDCs FFRDCs are subject to applicable direct competition limitations and cannot propose to this BAA in any capacity unless they meet the following conditions: (1) FFRDCs must clearly demonstrate that the proposed work is not otherwise available from the private sector. (2) FFRDCs must provide a letter on official letterhead from their sponsoring organization citing the specific authority establishing their eligibility to propose to Government solicitations and compete with industry, and their compliance with the associated FFRDC sponsor agreement’s terms and conditions.