HR001124S0033-Amendment-02.pdf

Broad Agency Announcement Intelligent Generation of Tools for Security

(INGOTS)

INFORMATION INNOVATION OFFICE

HR001124S0033

July 01, 2024

Amendment 02 As amended August 13, 2024

This publication constitutes a Broad Agency Announcement (BAA) as contemplated in Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016 and 2 CFR § 200.203. Any resultant award negotiations will follow all pertinent law and regulation, and any negotiations and/or awards for procurement contracts will use procedures under FAR 15.4, Contract Pricing, as specified in the BAA.

OVERVIEW INFORMATION:

Federal Agency Name – Defense Advanced Research Projects Agency (DARPA), Information Innovation Office

Funding Opportunity Title – Intelligent Generation of Tools for Security (INGOTS)

Announcement Type – Initial Announcement

Funding Opportunity Number – HR001124S0033

Assistance Listing Number: Not applicable

Dates/Time - All Times are Eastern Time Zone (ET) o Posting Date: July 01, 2024 o Proposal Abstract Due Date: July 18, 2024 at 1:00 p.m.

o Question Submittal Closed: August 13, 2024 at 5:00 p.m.

o Proposal Due Date: August 27, 2024 at 1:00 p.m.

Anticipated individual awards - Multiple awards are anticipated.

Types of instruments that may be awarded –Procurement contract or other transaction for prototype.

NAICS Code: 541715

Agency contact o Points of Contact

The BAA Coordinator for this effort may be reached at:

INGOTS@darpa.mil

DARPA/I2O

ATTN: HR001124S0033

675 North Randolph Street Arlington, VA 22203-2114

I. Section I: Funding Opportunity Description

The Defense Advanced Research Projects Agency (DARPA) is soliciting innovative proposals in the areas of vulnerability research and program analysis for system-centric security analysis, specifically focused on subjects related to the automation, synthesis, and analysis of exploit chains. Proposed research should investigate innovative approaches that enable revolutionary advances in the state of the art. Specifically excluded from this solicitation is proposed research that primarily results in incremental, evolutionary improvements to the existing state of practice.

Background: The INGOTS program was initially published June 23, 2023; however, during the scientific review process DARPA identified technical, structural, operational, and budgetary challenges which put the program, and any selected performer, at risk of failing. As such, DARPA has made programmatic and technical structure modifications to the INGOTS solicitation.

Program Objective: The INGOTS program will advance the state of the art in the field of vulnerability research for modern complex systems by developing novel techniques and technologies related to the creation, modification, modeling, and analysis of exploit chains, and on the use of exploit chains as tools for establishing system-centric assessments of security posture.

I.1. INTRODUCTION

As digital systems advance, they grow in complexity. Modern personal electronic devices typify this – mobile phones are systems of systems comprised of multiple discrete special-purpose hardware components, replete with a variety of defensive mitigations, which range from sandboxing to hardware-enforced memory protections. In such complex systems, holistic evaluation of security posture is increasingly difficult, and the assessment of isolated sub-components is often insufficient.

Exploit chains represent the state of the art in provably asserting the vulnerability of complex digital systems. Exploit chains combine multiple vulnerabilities within different system components to establish initial access, elevate privilege, and deliver effects, ultimately demonstrating the potential for an exploit to achieve a degree of control over a vulnerable system. This approach to exploitation is inherently system-centric, as exploit chains cross multiple execution contexts, abuse trust relationships within a system to bypass security boundaries, and leverage exploit primitives (specialized techniques) that are only viable in tandem with exploits or within a specific environment. Public disclosures of exploit chains, while still relatively uncommon, demonstrate the requisite level of complexity for attaining a meaningful degree of control (e.g., to enable persistence or root-level access) over a system.

Exploit chain compositions can range from as few as two discrete components to over a dozen and contain a combination of exploits and exploit primitives that satisfy constraints for chain components (e.g., bypass techniques for protections such as pointer authentication) or improve exploit reliability (e.g., inducing determinism in heap memory allocation patterns).

Today, the development of an exploit chain is a labor-intensive, manual process that requires a high level of domain expertise across multiple sub-disciplines within the field of vulnerability research. As such, the synthesis of a new exploit chain is often outpaced by the so-called “evergreen” model of modern software, firmware, and operating system updates.

This environment presents a challenge for those wishing to holistically understand the security posture of a given system. Existing exploit chains are frequently broken through updates to system components affected by or adjacent to exploit chains, and as is typical for any system, it is never immediately clear if developers sufficiently address the root cause of previously disclosed vulnerabilities in patches.

The goal of the INGOTS program is to address core limitations preventing the use of exploit chains as viable tools for evaluating the security of complex systems. INGOTS will achieve this goal through novel research into the creation, modification, modeling, and analysis of exploit chains, yielding new technologies and techniques that accelerate and automate the creation and maintenance of exploit chains, thereby enabling their timely application on a tempo that matches pace with the evolution of modern systems.

Success on INGOTS will require multiple advances in the state of the art in the fields of program analysis and vulnerability research. Program analysis techniques frequently grapple with the trade-off between precision in analysis and scale, whereas successfully automating analysis tasks associated with exploit chains will demand approaches that dynamically scale with system complexity. Furthermore, INGOTS will require a paradigm shift in the metrology surrounding vulnerabilities and weaknesses – Common Vulnerability and Exposures / National Vulnerability Database (CVE/NVD) entries currently represent severity in isolation. Exploit chains present a new context in which to consider the potential impact of a vulnerability on a given system, adding a useful dimension of complexity for the management of equities (such as prioritization of patching) surrounding vulnerabilities.

Upon completion, INGOTS will produce both advances in theory and in concrete application.

The program will also produce tools and technology that enable the synthesis, modification, modeling, and analysis of exploit chains. Ultimately, INGOTS will result in the creation of new technologies that shift exploit chains from the realm of the bespoke to that of the practical, enabling their everyday use as vital tools for vulnerability research on modern complex systems.

I.2. PROGRAM SCOPE

The INGOTS program will advance the state of the art in vulnerability research and program analysis through the lens of exploit chains. INGOTS is concerned with the security of complex systems, which cannot readily be assessed through the evaluation of components in isolation. Rather than perpetuate the current norm of reliance on qualitative measures of security properties, INGOTS seeks to accelerate the production and modeling of exploit chains as a means of providing empirical proof of system vulnerability and characterizing the impact that vulnerabilities can have on a given system.

To strike an appropriate balance between real-world-applicability and ease-of-development, INGOTS will focus on analysis of widely proliferated Android-based mobile devices, which make prominent use of open-source components. All facets/subcomponents of these devices, including cellular baseband, Wi-Fi, Bluetooth, browser, and operating system, are in-scope.

It is anticipated that a variety of exploit chains will be produced over the course of the INGOTS program, and that these exploit chains will, in aggregate, include coverage of the totality of major system subcomponents.

As INGOTS is focused on both the theoretical and practical aspects of exploit chains, fundamental research proposals are welcome but must either (1) have a means to deploy and integrate software into DoD systems without the need for developer access (e.g., via open-source libraries), or (2) have a teaming arrangement (either prime or subcontract relationship) that enables the performance of work at the Controlled Unclassified Information (CUI) and classified levels.

INGOTS will be predominantly concerned with N-day vulnerabilities in the first phases of the program, using publicly disclosed vulnerabilities as a source of requisite ground truth data for development and evaluation of theory and techniques. It is anticipated that as the program progresses and capabilities are developed and automated, 0-day vulnerabilities will become increasingly relevant to the program, and that discovery of such vulnerabilities may occur as a direct result of conducting program-sponsored research.

I.3. PROGRAM STRUCTURE

The INGOTS program is structured to address the unique challenges posed by advancing the state of the art in the creation and application of exploit chains through the following key technical areas (TAs):

TA1 – Exploit Chain Synthesis and Automation TA2 – Exploit Chain Modeling and Analysis TA3 – Test, Evaluation, and Integration

TA1 will center on topics surrounding the synthesis of exploit chains from their component exploits and primitives and will explore issues related to the formalization of exploit chain theory, the identification and characterization of exploits and primitives, and the automation of chain synthesis. TA2 will address the modeling and analysis of exploit chains and the systems they impact, exploring both the context and interdependencies defined within exploit chains, as well as in their relationship to a given system, and answer questions related to scale and mutability. TA3 will be dedicated to the test, evaluation, and integration of tools and techniques developed during research conducted across the program, and on ensuring the reproducibility of program-sponsored research.

Proposals to TA1 and TA2 must be independently executable. Proposals representing critical dependencies on other performers, technical areas, or government furnished information/equipment will not be considered. While it is expected that INGOTS performers will collaborate and integrate work between each other and across technical areas, planning to rely on external inputs does not constitute a sufficient capacity to perform the required work of this program.

I.4. PROPOSAL GUIDANCE

Abstracts will be required in advance of submitting a full proposal to this solicitation. Each abstract must be aligned to a single TA, but multiple abstracts may be submitted by a single organization to solicit feedback across multiple TAs. Each abstract must be separately submitted via Broad Agency Announcement Tool (BAAT). Upon receiving feedback encouraging a full proposal, the prime organization [1] may only submit a single proposal for a single TA, even if the organization has been encouraged on multiple abstract submission under multiple TAs. The submitted proposal must be based upon an encouraged abstract for that specific TA. Subcontractors shall only be proposed on a single proposal. An organization may submit a single proposal as a prime and be included in a single proposal as a subcontractor. If selected as a prime, the organization shall not perform as a subcontractor.

Proposers who submit multiple proposals or propose to multiple TAs in a single proposal will be considered non-compliant for the totality of their proposal submissions for this solicitation. Proposers who either do not submit an abstract or who receive feedback ‘discouraging’ a full proposal may not submit a full proposal. Please refer to Section III for further details regarding submission of both the abstracts and proposals.

As a means of risk reduction through a diversity of approaches, multiple awards are anticipated for TA1 and TA2. All proposers responding to this solicitation must provide a proposal encompassing a Phase I 12-month base period, and separately priced options for Phase II, Phase III, and a Transition Phase. It is possible that fewer performers may be funded to participate in later phases of the program. Determination to exercise options is at the sole discretion of the government, based on performance and funding availability.

To be considered eligible, proposers are required to include a resourced work breakdown structure aligned with the statement of work (SOW) and cost proposal as an additional artifact in proposal submission packages. This artifact is intended to help accelerate the completion of contract negotiation for selected proposals.

Funded performers are expected to collaborate with each other. The Government has determined that an Associate Contractor Agreement (ACA) is necessary to help facilitate an open exchange of information, to ensure compatibility between technologies developed under the INGOTS program, to prevent unnecessary duplication of effort, and to guarantee appropriate coordination and integration of work across TAs. All selected performers will be required to have their ACAs in place prior to the program kick-off meeting.

I.5. PROGRAM SCHEDULE

INGOTS is a 36-month program organized into three 12-month phases, with an optional 12-month Transition Phase to be funded by government partners. The 12-month optional Transition Phase will be concurrent with Phase III. The primary objectives for each phase will increase in complexity as the program progresses, building on the work of the prior phase. The fast-paced nature of the program will necessitate an environment of frequent

[1] An organization is defined as an entity such as a company, institution of higher learning, or an association.

collaboration, including the exchange of tools, techniques, and insights across all performers and technical areas. Each phase will have intermediate meetings, hackathons, and demonstrations, with opportunities for mid-point and end of phase evaluations.

Phase I (Base period, 12 months)

In Phase I, TA1 performers will focus on establishing the foundational theory for exploit chains, manually synthesizing exploit chains while extending the existing state of the art in automatic exploit generation to support automated generation of chain components or “links” (exploits and exploit primitives in a format suitable for use within exploit chains). TA2 performers will focus on modeling of exploit chain internals, enabling the logical decomposition of chains. Performers will also focus on analysis of exploits and exploit primitives to identify their features, constraints, and dependencies. TA3 performers will host two program hackathons and develop challenge problems based on publicly disclosed exploit chains for Android devices.

Phase II (Option period, 12 months)

In Phase II, TA1 performers will focus on exploring the mutability of exploit chains, developing techniques for automating the repair of exploit chains in the face of superficial changes (e.g., recompilation, library version updates) to affected systems of interest, as well as automating the replacement of exploit chain links as the result of substantive changes (e.g., patching of a vulnerability previously in use). TA2 performers will progress to scaling analysis, developing, and applying templates for the re-identification of exploit primitives across various corpora. Additionally, TA2 performers will develop system-centric models of exploit chains, characterizing the context and impact of exploit chains relative to the systems they affect. TA3 performers will continue to host program hackathons, expanding challenge problems to include variants of exploit chains created/analyzed by TA1/2 performers Phase I.

TA3 performers will also begin to integrate capabilities from TA1/2 performers and present the status and results of integration efforts to the government during program PI meetings.

Phase III (Option period, 12 months)

In Phase III, TA1 performers will focus on fully automating the synthesis of exploit chains, applying lessons learned from automated repair/replacement of exploit chain components to inform the generation of novel exploit chains “from scratch”. TA2 performers will leverage prior work into modeling and analysis to develop the concept of exploit chain forecasting, generating insights into the mutability, portability, and fragility of a given chain to aid in the planning and resource management needed to regularize the utilization of exploit chains in vulnerability research. TA3 performers will conduct final program hackathons, using real-world 0-day vulnerabilities and continue to present the government with integrated sets of capabilities developed under TA1 and TA2 efforts to prepare INGOTS tools and technology for transition to government partners.

Transition Phase (Optional) (Concurrent with Phase III)

All proposers to INGOTS TA1, TA2, TA3 should also include a 12-month Transition Phase option, which may be partially or fully funded by transition partners, for tasks related to supporting capability maturation, test, and evaluation with transition partners. Transition Phase development may involve further development of INGOTS technology, tailoring to specific requirements and/or operational needs which meet partner objectives. For the purpose of cost estimation, proposers should assume these Transition Phase options will be exercised during the execution of Phase III.

I.6. PROGRAM TECHNICAL AREAS

TA1 – Exploit Chain Synthesis and Automation

TA1 (Base) Scope

Exploit chains represent the edge of the art in provably demonstrating an exploit’s capacity to undermine the security posture of modern computer systems. Today, however, the system complexity inherently navigated by (and to an extent, encapsulated within) exploit chains present a challenge beyond the capacity of existing tools and techniques to meaningfully automate. This has ensured that the development of exploit chains remains high cost in terms of expertise, labor, and time. INGOTS TA1 will focus on advancing the state of the art in vulnerability research through developing sufficient understanding, techniques, and automation technologies to commoditize exploit chains.

INGOTS TA1 performers will research and develop capabilities for automated generation of exploit chains and their component exploit and exploit primitives. While Android mobile devices will be the primary focus of INGOTS, it is anticipated that most of the theory, techniques, and technologies developed during the program will be generalizable to other classes of systems of interest. Over the course of the program, performers will move from expanding upon existing studies and practices in automatic exploit generation to the development of novel techniques for coherently combining sets of exploits and exploit primitives into viable exploit chains.

TA1 performers will be responsible for establishing norms within the vulnerability research community related to the theory and practice for exploit chains, their composition, synthesis, and maintenance. Strong proposals will highlight the proposer’s capacity to engage with the greater security community and advance the public narrative on the topic of exploit chains.

Relevant evidence may include a track record of academic publications, open-source software contributions and usage statistics, industry conference presentations, etc.

Performers on TA1 shall conduct research and development of automated techniques for the discovery and tailoring of new exploit primitives that are composable with exploits. Strong proposals will present a discussion on existing disaggregated tools and techniques related to exploit primitives and present a path to a unified approach for managing their dependencies, constraints, and capabilities. Strong proposals will also present solutions for enabling the programmatic identification and incorporation of relevant exploit primitives into exploit chains.

TA1 performers will also be responsible for the expansion of existing approaches to automatic exploit generation to yield exploit variants which support programmatic integration into exploit chains. Strong proposals will highlight relevant past performance in the field of automatic exploit generation and discuss shortcomings of current automatic exploit generation (AEG) technologies and propose solutions for overcoming existing issues, especially those related to scale and complexity.

Performers on TA1 shall conduct research and development of automated techniques for the reconstitution of exploit chains through repair and replacement of component exploits and exploit primitives. Strong proposals will demonstrate a clear understanding of the difference in situations and effort required for repair versus replacement. In addition to proposed solution implementations, strong proposals will present solutions for assisting human maintainers in the correct identification of repair or replacement scenarios.

Most importantly, INGOTS TA1 performers will develop toolchains enabling the automated synthesis of novel exploit chains capable of achieving a variety of effects on devices of interest. Strong proposals will illustrate how a sufficiently mature understanding of the subject will yield meaningful progress on automation and discuss potential pitfalls and mitigation strategies to ensure that such automation technology is realized prior to program completion.

It is anticipated that a unique combination of expertise is required to successfully achieve the ultimate goal of TA1 and create fully automated exploit chain synthesis technology. Strong TA1 proposals will highlight a team comprised of both expert practitioners in the field of vulnerability research as well as developers with a wealth of experience in automation.

Strong proposals will also present evidence of successful past performance in development, utilization, and enhancement of automation engines for vulnerability research, such as automated exploit generation frameworks, Return-Oriented-Programming (ROP) compilers, or static code analysis engines capable of vulnerability discovery at scale.

TA1 Challenges and Approaches

Over the course of research and development in INGOTS TA1, performers are expected to materially advance the state of the art in the field of vulnerability research. To enhance the government’s confidence in proposed solutions, all proposers are strongly encouraged to consider and, when appropriate, include exposition related to the anticipated challenges and potential approaches highlighted below.

Maturity in Software Development Practices and Efficient Collaboration – Throughout the program, TA1 performers will instantiate and enhance tools to automate the development and synthesis of exploit chains. Simultaneously, other performers (especially those in TA3) will depend on the ability to access, utilize, and build upon these tools. Performers will be expected to define and deliver a minimum viable product for solving TA1 challenges and demonstrate the ability to consistently iterate and deliver functional improvements at a high cadence. Strong proposals will present past performance that demonstrates the ability to leverage modern software development practices (e.g., Infrastructure-as-Code, DevOps, regular releases, etc.) and enable high-tempo technical collaboration with both external teams and individual contributors, including demonstrable evidence of setting and meeting target development goals at regular intervals.

Vulnerability Knowledge Sourcing and Management – While it is certainly true today that several datasets, such as CVE/NVD, attempt to capture knowledge of vulnerabilities at scale, prior research has shown that such datasets suffer from uneven information density and rarely provide sufficient information on their own to programmatically confirm that a given vulnerability has been re-encountered (i.e., via automated vulnerability discovery tools).

Beyond these datasets, a variety of other data types can also serve as viable sources of information about vulnerabilities, including system crashes, upstream releases, patch notes, blog posts, public exploit disclosures, etc. Strong proposals will present novel solutions for effectively satisfying the need for knowledge of vulnerabilities as a prerequisite for identifying the possible permutations of exploits and exploit primitives to include in an exploit chain. Strong TA1 proposals will also discuss methods for sourcing a variety of artifacts (e.g., web scrapers, named entity extraction), striking a balance between the quality and quantity of such artifacts, managing the organization of these artifacts, and distributing this information for collaboration and automation though the use of documented APIs or data formats.

Achieving Automatic Exploit Primitive Generation – For the purposes of INGOTS, exploit primitives can best be equated to simple weird machines (or emergent execution engines), that provide exploits with a limited degree of control within a given context.

Exploit primitives do not yield arbitrary code execution on their own, but are often essential to optimizing exploits (e.g., enabling privileged payload execution with a single byte arbitrary write) or ensuring exploit reliability (e.g., via any number of existing methods for inducing determinism in heap allocation patterns). Historically, the discovery and utilization of exploit primitives has been a largely manual process. Recently however, attempts at systematization have begun for certain classes of exploit primitives, such as kernel heap manipulation. Strong TA1 proposals will present approaches to systematizing the discovery and characterization of exploit primitives and suggest how to prioritize research into additional classes of exploit primitives. Although scaling analyses across corpora of software and firmware to confirm the existence of exploit primitives in other systems is firmly within the realm of TA2, strong TA1 proposals will also discuss the interplay of proposed methods for characterization or templating of exploit primitives with potential TA2 analysis approaches, but should not make their TA1 approaches dependent upon TA2.

Maturing Automatic Exploit Generation – Given a source of vulnerability knowledge, AEG has been proven viable for certain classes of vulnerabilities. It is anticipated that INGOTS will significantly contribute to maturing techniques for AEG research into automating exploit chain synthesis. Strong TA1 proposals will identify limitations endemic to existing AEG approaches and present opportunities for advancing beyond such deficiencies. Strong proposals may choose to discuss the ramifications of logically de-coupling exploit primitives from exploits, or how AEG feasibility changes with variance in the complexity of exploit chains.

Exploit Chain Mutability, Component Repair and Replacement – Exploit chains exist within dynamic systems, yet are highly-dependent on the specific software, firmware, and hardware configurations they affect. As such, exploit chains are inherently fragile, with a complex set of dependencies that arise from each component exploit and exploit primitive.

Exploit chain mutability then is the potential for the reconstitution of a functional exploit chain through the modification or substitution of one or more components. That components will be rendered non-viable when system changes occur is a reality faced by all exploit chain maintainers, and strong TA1 proposals will illustrate solutions for minimizing the time-to-reconstitution of an exploit chain through automation. Strong proposals will also discuss piecewise component repair within the context of AEG (e.g., reprovisioning exploits via modified constraints) and provide special consideration for how exploit primitives are commonly impacted by system changes. Strong proposals will also present the automation of component replacement as a sub-problem of automatic exploit chain synthesis and explore both how to identify the minimal set of components to be replaced as well as how to generate replacement components that conform to the constraints and dependencies of the remaining legacy chain components.

Automating Exploit Chain Synthesis – Although the characterization of exploit chains is made apparent through their use of multiple exploits and exploit primitives in tandem, their true defining feature is the effect they ultimately achieve, such as the ability to persist across reboots, to gain access to a particular application’s database, or to root a device. The constitution of an exploit chain is entirely driven by the need to establish sufficient access and privilege to achieve the desired effect. This goal-centric approach presents a basis for how TA1 performers may choose to approach automation of exploit chain synthesis, and strong TA1 proposals will discuss how the proposed automation methods either embrace or eschew existing goal-oriented automation techniques. Additionally, strong proposals will discuss how their approach to automate synthesis actively addresses the complexity (e.g., through summarization/abstraction, piecewise construction) of managing dependencies, constraints, and capabilities of multiple exploits and exploit primitives.

TA2 - Exploit Chain Modeling and Analysis

TA2 (Option) Scope

Exploit chains have been a topic within vulnerability research for over a decade, entering mainstream discourse from the iPhone jailbreak community. Although the concept of an exploit chain is widely understood, little standardization or formalism has been achieved on the topic, leading to inconsistent treatment and presentation. INGOTS TA2 will therefore focus on development and implementation of foundational models and analytics for exploit chains, expanding existing vulnerability research and program analysis techniques to exploit chain synthesis, refinement, and forecasting.

INGOTS TA2 will establish the foundational concepts and models for defining and describing the composition and impact of exploit chains. Although exploit chains are an increasingly common subject in the vulnerability research community, there is still little consensus on how to present more detailed explorations of exploit chains. TA2 performers will be responsible for conducting modeling research to support the establishment of new norms and formalisms that aid in the development, analysis, and refinement of exploit chains.

Strong TA2 proposals will discuss the existing body of relevant technical write-ups, conference presentations, and academic papers when presenting their approach to moving beyond the current superficial representation of exploit chains as merely “multiple exploits used in tandem”. TA2 models should support the detailed interrogation of exploit chains and their component parts – proposed approaches that are limited to decomposition of chains into a sequence of exploits will be considered non-responsive.

TA2 performers should present modeling approaches that support both ingest of existing exploit chains as well as aid in the synthesis of new or refined exploit chains. Strong TA2 proposals will highlight the proposer’s capacity to engage with the greater security community and advance the public narrative on the topic of exploit chain modeling. Relevant evidence may include a track record of academic publications, industry conference presentations, etc. Strong proposals will present a coherent strategy for improving shared understanding of exploit chain modeling approaches across the vulnerability research community as the state of the art continues to evolve though program-sponsored research.

TA2 performers will research and develop analytic capabilities that enable the interrogation of exploit chain internals to recover their function and composition. Today, exploit chain analysis is a largely manual process, and is focused on component exploits as the mechanisms of action. TA2 performers will develop capabilities that support automation of exploit chain analysis and incorporate the roles of all chain components, including exploit primitives and effects payloads. Strong proposals will present an approach to analysis research and development that identifies specific insights into function or composition that may be gleaned from specific exploit chain components. As exploit chains span multiple components within affected systems, research into improving or tailoring the existing state of the art in multi-binary program analysis is expected to be necessary. Strong proposals will both discuss required research into analytics and present a strategy for evaluation of analysis efficacy. TA2 proposals that only intend to research analysis approaches for exploit chains and do not specifically include plans for implementation and delivery of analytic prototypes will be considered non-responsive.

Performers on TA2 will work to establish a state of the art in analysis of exploit chains within the context of affected systems. Current norms within the security community rely on qualitative severity metrics. However, such metrics oversimplify the exploitability and impact of a vulnerability by ignoring both the context in which an exploit might be employed (e.g., as part of an exploit chain) as well as the environmental realities of the affected systems. TA2 performers will therefore research and develop methods for system-centric quantitative analysis of exploit chains. These analyses will focus on determining the characteristics of both exploit chain components as well as related system components (e.g., vulnerabilities, trust relationships between system components). Strong proposals will also present a system to provide improved impact assessment and prioritization of vulnerabilities using quantitative metrics, such as the prevalence of a vulnerability or exploit primitive across software versions. Although INGOTS is primarily focused on Android mobile devices, strong proposals will also present solutions that are generically applicable to the range of systems potentially affected by exploit chains.

INGOTS TA2 performers will also work to establish a science of exploit chain forecasting, based on insights derived from improved modeling and analysis capabilities. Performers will develop tools to support assessments of the fragility and generalizability of exploit chains through analysis of existing exploit chain components, as well as via identification and assessment of alternative components. Strong TA2 proposals will discuss the utility and feasibility of forecasting in an environment of frequent change, and present approaches that enable researchers to efficiently align resources to prioritized aspects of exploit chains (e.g., patching a vulnerability in which multiple exploit chains converge). Strong TA2 proposals will also present solutions to manage the complexity and scaling required for accuracy in forecasting.

TA2 Challenges and Approaches

Over the course of research and development in INGOTS TA2, performers are expected to substantively advance the state of the art in the field of vulnerability research. To enhance the government’s confidence in proposed solutions, all proposers are strongly encouraged to consider and, when appropriate, include exposition related to the anticipated challenges and potential approaches highlighted below.

Exploit Chain Foundational Models - The Common Weakness Enumeration (CWE) and Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK) frameworks have established ontological consensus for the classification of vulnerabilities and offensive cyber tactics, techniques, and procedures. However, these frameworks lack sufficient concepts and language to fully enable advancements in the research and analysis of exploit chains. This status quo has resulted in discrepancies in how various researchers describe the function, classification, and piecewise construction of exploit chains, especially relative to an affected system’s composition and boundaries. Strong TA2 proposals will discuss their approach to exploit chains modeling by presenting an initial ontology, based on current literature and knowledge of existing frameworks. Strong proposals will also highlight how models can assist not just in improving the ability to interrogate existing exploit chains, but also augment synthesis and refinement. As the topic is explored across the program, strong proposals will discuss how such features will provide useful tooling to help meet TA1 objectives. Strong proposals will also present a strategy for building consensus on exploit chain modeling within the security community, such as how specific features might help spur more widespread model adoption beyond the confines of the program.

Compositional Analysis –To ensure that models developed under INGOTS are of practical use, TA2 performers will design and implement analytic models to enable the recovery of exploit chain structure. The synthesis of exploit chains from components requires that authors navigate a variety of pre-conditions, constraints, and post-conditions for each exploit, primitive, or effect. Analyses which enable vulnerability researchers to recover such information will assist in both maturing TA1 research into exploit chain synthesis, as well as accelerate the current manual, labor-intensive process by which exploit chains are analyzed.

While it is expected that TA2 performers will research and develop automated methods such as fingerprinting for exploit chain components, strong proposals will also present solutions for automating the discovery and characterization of the interrelationships between components. Strong proposals will also present a discussion on the recovery of exploit primitives, identifying appropriate program analysis techniques and presenting a brief case study (e.g. differentiating between different heap grooming techniques). Strong proposals will also discuss discovery and characterization of constraints incurred through chaining components, including how to identify if such constraints are inherent limitations due to exploitation of vulnerabilities or arise from the chosen exploit chain composition.

Functional Analysis – As previously stated, the true defining feature of an exploit chain is the effect ultimately achieved. However, when analyzing exploit chains, understanding how an effect is achieved is frequently of equal importance, such as when determining the mutability of an exploit chain, evaluating an exploit chain’s potential application to different systems, or in identifying critical components of the affected system to prioritize patching.

INGOTS TA2 performers will therefore also develop a set of functional analyses that enable the identification of both the ultimate effect of an exploit chain as well as the sub-goals or objectives along the path of exploitation. Strong TA2 proposals will present a set of relevant sub-goals and end states to recognize and characterize, such as inducing determinism in memory allocation, gaining arbitrary code execution, the exposure of additional attack surfaces, privilege escalation, etc. Strong proposals will also discuss how to recover the functionality of exploit chain components while minimizing human-provided semantic context and identify opportunities for automation. Strong proposals will also discuss the relationship between exploit chain compositional and functional analysis, e.g., how component characterization may help provide semantic context for automating functional inference.

Analysis at Scale – Exploits and exploit primitives are frequently applicable to a plurality of software and hardware, as is evidenced by the Common Platform Enumeration (CPE) entries associated with CVEs. Although vulnerability disclosures attempt to exhaustively enumerate impacted platforms, they do so without exhaustive empirical testing, resulting in the chronic under-reporting of affected platforms. Furthermore, such impact assessments are limited only to vulnerabilities – no widely adopted framework or database attempts the aggregation of exploit primitives, or reports on the platforms in which such primitives are present. INGOTS is concerned with the accurate representation of both vulnerabilities as well as exploit primitives for enabling characterization of the mutability or fragility of exploit chains. As such, TA2 performers will be required to conduct analysis at scale over large corpora of software and firmware, focused on the re-identification of known vulnerabilities and exploit primitives. Strong proposals will present mature automated tooling approaches to effectively scale analysis and present evidence of successful past performance in achieving scale on similar program analysis problems (e.g., static code analyzers, disassemblers, decompilers).

Strong TA2 proposals will also discuss how analysis at scale can enhance TA1 exploit chain mutability assessments, or how improving knowledge of vulnerabilities and exploit primitives at scale can impact potential approaches to the synthesis of new exploit chains.

System-Centric Exploit Chain Analysis – Today, vulnerabilities are common – the sheer volume of public disclosures in the NVD provides ready proof of this fact. However, the NVD’s focus on the identification and characterization of individual vulnerabilities also includes a qualitative assessment of vulnerability severity, the Common Vulnerability Scoring System (CVSS). CVSS scores are by necessity qualitative, as the score is typically derived absent the full context of use in an exploit chain against specific systems. This unfortunately results in “hard-shell” defensive prioritization in which patch prioritization focuses on perimeter vulnerabilities and risks inadequate prioritization for remediating the (potentially more widely exploited) vulnerabilities that are not immediately accessible from outside of system security boundaries. INGOTS seeks to commoditize the use of exploit chains as tools for wholistically evaluating system security. In support of this objective, TA2 performers will research and develop analytic capabilities that can identify and characterize the interactions of an exploit chain with affected systems. Strong TA2 proposals will present a system-centric model of exploit chain interactions, along with a set of proposed supporting analyses which enable researchers to effectively navigate the relationship between exploit chain and affected system (e.g., investigate exploited system contexts). Strong proposals will also address generalization beyond Android mobile devices and discuss how the offered solution has the capacity to work at the scale and diversity of the NVD/CPE.

Exploit Chain Forecasting – Exploit chains exist in an ever-changing ecosystem, where shifts in the terrain (e.g., patching or modification of software within affected systems) are common occurrences that can result in either superficial or critical damage to chain integrity.

Despite special cases when information related to impending change can be known (such as when knowledge of upstream patches and release schedules are available), the frequency and magnitude of modification to the systems affected by a given exploit chain cannot typically be reliably predicted. As the INGOTS program progresses, it is expected that TA2 modeling and analysis efforts as well as TA1 mutability assessments will mature that understanding of exploit chains, presenting an opportunity to conduct forecasting by leveraging a combination of enhanced knowledge of exploit chains and historical information for affected systems in lieu of exquisite knowledge of future system releases. TA2 performers will therefore leverage capabilities created in earlier phases of the program to develop a set of capabilities focused on forecasting for particular exploit chain attributes (e.g., such as localizing fragility to specific components). This forecasting is intended to enable the efficient allocation of resources by researchers, e.g., for efforts related to maintenance of chains or to further enhance the prioritization of patches (especially for vulnerabilities or primitives that exist within the security boundary). Strong TA2 proposals will discuss solutions for managing the complexity associated with the volume and diversity of information required for forecasting.

Strong proposals will also present solutions for prioritizing patching or exploit chain component replacement strategies which make use of such forecasting data.

TA3 – Test, Evaluation, and Integration

TA3 (Option) Scope

Bridging the chasm between research prototype and operational capability requires a degree of refinement that is frequently absent from motivations behind open-source or public vulnerability research. While proofs of concept are suitable for demonstrating the existence of exploitable flaws in system components, INGOTS strives to use exploit chains to attain a more holistic assessment of a system’s security posture, and therefore requires a deeper understanding than what is offered by a typical PoC. This insight can only be attained through a more thorough exploration which includes the evaluation and refinement (e.g., to establish realistic reliability metrics) of exploit chains and their components. Performers on INGOTS TA3 will establish the necessary infrastructure to conduct the test and evaluation of TA1/TA2 capabilities in support of this goal and will help guide program research through creation of hackathon challenge problems. TA3 performers will also be responsible for integrating mature tools developed by performers in other TAs and work closely with the Government Team to prepare prototypes for transition.

Modern development and testing of exploit chains is an imprecise and, at times, probabilistic science. Exploit test and evaluation systems vary from low-cost emulators to intricately crafted hardware suites representing spectrums of make, model, and version. In both cases, testing is laborious and inexhaustive, limiting the reliability of outcomes. This results in a status quo where qualitative metrics are used to approximate vulnerability impact based on simplistic heuristics rather than quantitative and comprehensive analyses. Real systems are rife with intricacies and complexity, which threaten to obscure the assessment of exploit and exploit chain impact and performance. INGOTS TA3 will focus on enabling the quantitative evaluation of exploit chains through the development of automated standardized tools. Strong proposals will present a series of quantitative metrics and discuss both the rational for such metrics and plans for implementation. Strong proposals will also account for the incorporation of modeling and analysis tools written by TA2 and anticipate the potential for such tools to support TA3 objectives.

TA3 will be responsible for development of environments and tooling for automating the test and evaluation of exploit chains and their component exploits and exploit primitives. Strong proposals will present a discussion of existing test environments, focusing on the state of the art in Android or other mobile phone vulnerability research. Strong proposals will also discuss the application of automation to existing tools and techniques related to exploit chains evaluation and present a plan for overcoming existing drawbacks to current tools and techniques when applying them to exploit chains.

The TA3 performer will continuously test and evaluate technology produced by TA1 and TA2 for transition suitability, and work to integrate INGOTS technologies and support transition to Government partners. TA3 will be responsible for leading the development of the required Associate Contractor Agreement (ACA), in close collaboration with all other performers.

TA3 will work closely with the Government team to help guide and align research objectives on TA1 and TA2 through a series of hackathons. During each phase of the program the TA3 performer will be responsible for organizing two (2) hackathons and developing a set of challenge problems to focus the research and development activities during these events.

Strong proposals will present evidence of successful past performance in the development of challenge problems for other hackathons, Capture-the-Flag contests, or similar events. Strong proposals will also discuss how planned challenge problems can progress in difficulty and scope to align with the TA1/TA2 technical objectives previously outlined in this document.

The INGOTS TA3 performer will assist the Government team in transition activities including demonstration/hackathon coordination and meeting with transition partners. To support this role, the TA3 team must include personnel with a TS/SCI clearance – at a minimum the PM and PI, though more are encouraged. Strong proposals will briefly discuss past transition activity support and work with potential transition partners on similar programs.

TA3 Challenges and Approaches

Testbed Development - Today, empirical testing remains the most reliable means for evaluating the performance of a cyber capability, whether exploit, primitive, or chain.

However, such testing and evaluation is inherently limited: based on the fidelity of instrumented environments, availability of a variety of representative system configurations, and the…