HR001120S0032-Amendment-01.pdf

HR001120S0032

Broad Agency Announcement Data Protection in Virtual Environments (DPRIVE)

Microsystems Technology Office

HR001120S0032

February 27, 2020

Amendment 1 As amended March 19, 2020

Amendment 1 (1) clarifies that DPRIVE solutions should support 2-3 FHE algorithms which must at a minimum include BGV, and (2) details the opportunity for proposers to use Air Force Research Laboratory’s Palladium design and emulation tools.

FOREWORD

In June 2017, DARPA announced the Electronics Resurgence Initiative (ERI) as a bold response to several critical emerging trends. Among them, the cost and complexity of advanced microelectronics design and manufacture has increased rapidly, challenging Gordon Moore’s economic premise that future machines would require lower costs and shorter turnaround times.

For the first time, the defense community faces few or no options for accessing leading-edge electronics, the result of cost-driven foundry consolidation. Meanwhile, non-market foreign forces are working to shift the electronics innovation engine overseas, challenging U.S.

economic and security advantages. In addition, the nation is gaining a new appreciation for electronics security—a longtime defense concern—following publicized challenges to our digital backbone in sectors as diverse as automotive, cybersecurity, and voting. ERI envisioned a unified national response marked by research collaborations between DARPA, the defense community, academia, and the commercial sector.

There is significant historical precedent to suggest the viability of this approach; each wave of modern electronics development has benefitted from the combination of defense-funded academic research and commercial sector investment. In the 1980s, when geometric scaling started to make low-volume integrated circuit fabrication unaffordable, DARPA’s investment in the Metal Oxide Silicon Implementation Service (MOSIS) opened the door to rapid, low-cost chip manufacture, laying the foundation for the nation’s world-leading fabless design industry. In the 1990’s, a combination of defense, academic, and commercial partners pioneered 193 nm lithography, which became the industry-critical fabrication process. Then, as Dennard scaling ended in the 2000’s, the semiconductor industry adopted Fin Field Effect Transistors (FinFETs), another DARPA-funded innovation that drove to low power computing and led to the era of 3D devices.

The nation now stands ready to innovate a 4th wave of electronics progress. The state of the industry indicates that the 4th wave will be defined by three-dimensional heterogeneous integration. Through integration, innovators will add new materials and devices to the silicon foundation and enable intelligence and specialized functions precisely designed to meet the diversifying needs of the commercial and defense sectors. 3D heterogeneous integration will also demand new architectures and design tools, developed to manage the complexity of working in three dimensions while enabling rapid system upgrades and integrating security as a primary design concern. These areas—3D heterogeneous integration, new materials and devices, specialized functions, and design and security—have been central to ERI since its inception and will continue to guide the initiative as it enters its third year.

The 4th wave of electronics progress, however, is neither inevitable nor inevitably beneficial to the United States. As a community, the collective challenge faced by DARPA and its ERI partners will be to ensure that benefits differentially accrue to the U.S. commercial and defense base, which is aggressively investing in continued progress. To meet our national security needs, 4th wave technologies must enable more capable systems that process data locally, extract actionable information, and make decisions at “the edge”. To address new security concerns, 4th wave technologies must integrate security considerations into microsystem design in a way that is both effective and easy to implement. To address the rapid rise of devices operating at the edge, the glut of information those devices will collect, and the growing cyber-driven threats those devices will encounter, 4th wave technologies must find ready adoption by the commercial and defense sectors. New and existing ERI programs will therefore increasingly address the challenge of transitioning 4th wave technologies to the domestic sectors that need them.

Together with the ongoing ERI programs, the program addressed in this Broad Agency Announcement (BAA) will continue to provide a foundational contribution both to U.S. national security and to the needs and ambitions of the domestic commercial sector. DARPA seeks to receive proposals from entities that can help to achieve this goal. For reference, an updated list of ERI programs, solicitations, and events is available via https://www.darpa.mil/work-with-us/electronics-resurgence-initiative.

https://www.darpa.mil/work-with-us/electronics-resurgence-initiative https://www.darpa.mil/work-with-us/electronics-resurgence-initiative

Table of Contents

PART I: OVERVIEW INFORMATION

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description A. Background B. Program Description C. Program Structure D. Technical Area E. Schedule/Milestones F. Deliverables G. Commercialization H. Government Furnished Equipment/Property/Information I. Intellectual Property

II. Award Information A. General Award Information B. Fundamental Research

III. Eligibility Information A. Eligible Applicants

1. Federally Funded Research and Development Centers (FFRDCs) and Government Entities

B. Organizational Conflicts of Interest C. Cost Sharing/Matching D. Other Eligibility Criteria

1. Collaborative Efforts IV. Application and Submission Information

A. Address to Request Application Package B. Content and Form of Application Submission

1. Abstract Format

2. Full Proposal Format

3. Proprietary Information

4. Security Information

a. Program Security Information

b. Unclassified Submissions

c. Classified Submissions

5. Disclosure of Information and Compliance with Safeguarding Covered Defense

Information Controls

6. Human Subjects Research (HSR)/Animal Use

7. Approved Cost Accounting System Documentation

8. Section 508 of the Rehabilitation Act (29 U.S.C. § 749d)/FAR 39.2

9. Small Business Subcontracting Plan

10. Intellectual Property

a. For Procurement Contracts

b. For All Non-Procurement Contracts

11. Patents

12. System for Award Management (SAM) and Universal Identifier Requirements

13. Funding Restrictions

C. Submission Information

1. Submission Dates and Times

a. Abstract Due Date

b. Full Proposal Date

c. Frequently Asked Questions (FAQ)

2. Abstract Submission Information

3. Proposal Submission Information

a. For Proposers Requesting Technology Investment Agreements

b. For Proposers Requesting Contracts or Other Transaction Agreements

c. Classified Submission Information

4. Other Submission Requirements

V. Application Review Information A. Evaluation Criteria

1. Overall Scientific and Technical Merit

2. Potential Contribution and Relevance to the DARPA Mission and Technology

Transition

3. Cost and Schedule Realism

B. Review and Selection Process

1. Review Process

2. Handling of Source Selection Information

3. Federal Awardee Performance and Integrity Information (FAPIIS)

VI. Award Administration Information A. Selection Notices

1. Abstracts

2. Proposals

B. Administrative and National Policy Requirements

1. Meeting and Travel Requirements

2. FAR and DFARS Clauses

3. Controlled Unclassified Information (CUI) on Non-DoD Information Systems

4. Representations and Certifications

C. Reporting D. Electronic Systems

1. Wide Area Work Flow (WAWF)

2. i-Edison

3. TFIMS

VII. Agency Contacts VIII. Other Information

A. Proposers Day B. Protesting

ATTACHMENT 1: Cost Volume Proposer Checklist ATTACHMENT 2: Proposal Summary Slide Template

PART I: OVERVIEW INFORMATION

Federal Agency Name: Defense Advanced Research Projects Agency (DARPA), Microsystems Technology Office (MTO)

Funding Opportunity Title: Data Protection in Virtual Environments (DPRIVE) Announcement Type: Initial Announcement Funding Opportunity Number: HR001120S0032 Catalog of Federal Domestic Assistance Numbers (CFDA): Not applicable Dates: (All times listed herein are Eastern Time) o Posting Date: February 27, 2020 o Proposers Day: March 2, 2020 o Abstract Due Date: March 23, 2020 o FAQ Submission Deadline: May 16, 2020 o Proposal Due Date: June 2, 2020 o Estimated period of performance start: November 2020

Concise description of the funding opportunity: DARPA is soliciting innovative research and development to reduce the computation run time of fully homomorphic encrypted computation through hardware acceleration. The goal of the DPRIVE program is to enable fully homomorphic encrypted computation to within one order of magnitude of the compute time of current unencrypted computation.

Anticipated Funding Available for Award: Total award funding is expected to be $33M.

Anticipated individual awards: Multiple awards are anticipated.

Anticipated funding type: 6.2 Types of instruments that may be awarded: Procurement contract or other transaction Agency contact:

o Dr. Tom Rondeau, Program Manager BAA Coordinator: DPRIVE@darpa.mil

DARPA/MTO

ATTN: HR001120S0032

675 North Randolph Street Arlington, VA 22203-2114 mailto:name@darpa.mil

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

The Defense Advanced Research Projects Agency (DARPA) often selects its research efforts through the Broad Agency Announcement (BAA) process. This BAA is being issued, and any resultant selection will be made, using either the procedures under Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016 and 2 C.F.R. § 200.203 or Other Transactions (OT).

Any negotiations and/or awards will use procedures under FAR 15.4, Contract Pricing, or OT as applicable. Proposals received as a result of this BAA shall be evaluated in accordance with evaluation criteria specified herein through a scientific review process.

DARPA BAAs are posted on the Beta SAM website, under the Contract Opportunities (FBO) link, at https://beta.sam.gov/. The following information is for those wishing to respond to the

BAA.

The Microsystems Technology Office at DARPA seeks innovative proposals in the research, design, and implementation of fully homomorphic encryption (FHE) processing hardware that enables FHE computation at time scales that are within an order of magnitude of unencrypted computation on conventional CPU-based computing platforms. DPRIVE will enable the use of fully homomorphic encryption to protect data during computation as well as during storage and transmission. The outcome of DPRIVE will be a fabricated chip and supporting hardware/software that realizes the goals of the program. The program will have a single technical area. BAA submissions must fully address all technical challenges discussed below.

Proposals that do not address all technical challenges will be considered non-conforming.

Proposed research should investigate innovative approaches that enable revolutionary advances in science, devices, or systems. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice. Given the anticipated challenges of the DPRIVE program, multidisciplinary teams with grounded experience in the mathematics of FHE, embedded software systems, and unconventional VLSI hardware design are encouraged to submit proposals.

A. Background

Current art in cryptographic solutions, such as Advanced Encryption Standards (AES), provide security of data while it is in transit across a network or at rest while in storage. Computations that manipulate data in useful ways (i.e., transaction processing, analytics, machine learning, etc.) require the data to be decrypted. Such decryption exposes the data to local processes and other vulnerabilities, e.g., zero-day attacks. Current encryption therefore does not allow full data protection.

A known solution to this problem is fully homomorphic encryption, which enables computation on ciphertext (encrypted data) rather than plaintext (unencrypted data). The key enabling concept of FHE is lattice cryptography, which presents complex mathematical challenges to the adversary that, without knowledge of the secret structure (i.e., the key), are well beyond the capability of current technology including theoretical quantum computers to solve. Critically, https://beta.sam.gov/ lattice encryption enables mathematical operations that work directly on the ciphertext; these same operations applied to the underlying plaintext produce the same numerical result, thereby enabling secure computation.

The FHE concept is illustrated in Figure 1, where it is compared to traditional encryption.

Figure 1: Current “secure” computation versus FHE computation

Intrinsic to lattice encryption is an unavoidable accumulation of noise with every computation.

Noise accumulation rapidly rises above a threshold where recovery of the underlying plaintext becomes impossible. The “bootstrapping” innovation reduces accumulated noise down to a level comparable to the original ciphertext but produces orders of magnitude more overhead in the computations (bootstrapping is further discussed in the Program Description below).

The increase in FHE’s computation overhead arises from the explosion in the number of computations which must take place, the size of the data vectors used in each computation, and limitations in current processing hardware due to inherent restrictions in architecture and data representation. A single bootstrapping operation in FHE can require numerous multiply and modulo operations of data with word sizes of 1000s of bits.

Word size directly relates to the signal-to-noise ratio (SNR) of how a ciphertext is stored and manipulated in computation. Recent studies demonstrate that using large arithmetic word size (LAWS) increases SNR in the FHE computations. Larger SNR results in reduced accumulation of noise at each compute step and reduced overhead burden from costly operations such as bootstrapping. Figure 2 illustrates the potential reduction in FHE compute runtime by moving to a LAWS data representation.

Figure 2: Word size directly relates to computational noise and how ciphertext is stored and manipulated.1

Current processor architectures are based on data representation of 64 bits or less. Virtualization of larger bit word sizes are possible, but ultimately when processed through an arithmetic logic unit (ALU), the data is reduced to native architecture word size of 64 bits or less. The DPRIVE program seeks to build a hardware accelerator that can realize the potential reduction on compute time for FHE by natively processing on LAWS operands of 1024 bits or more. Developing a hardware accelerator to natively process LAWS operations introduces challenges and opportunities for additional gains in compute runtime through memory management and flexible data structure, as detailed below in the Program Description.

B. Program Description

The DPRIVE program will develop a hardware accelerator for FHE that reduces computational run-time overhead by many orders of magnitude compared to software-based FHE computations on conventional CPUs. FHE enables operation on data in the encrypted state, providing a much higher degree of security as compared to conventional encryption methods where decryption is required to operate on the data. Implementation of FHE is currently impractical due to the enormous computation time required to perform even simple operations. Through new insights into computational structures that minimize the effects of an FHE processing procedure known as bootstrapping2, DPRIVE will speed up FHE calculations to within one order of magnitude of the current CPU performance of computations on unencrypted data, making FHE practical and more widespread for many applications (Figure 3).

1 Analysis performed by Kurt Rohloff (NJ Institute of Technology), DARPA Young Faculty Awardee.

2 Craig Gentry. “Fully Homomorphic Encryption Using Ideal Lattices,” 41st ACM Symposium on Theory of Computing (STOC), 2009.

Figure 3: DPRIVE accelerators will establish the foundation for FHE computation with minimal run-time burden (10x over plaintext).

To improve FHE computation speeds, the DPRIVE program will address the debilitating overhead required for FHE computation. The key insight to the program is that Large Arithmetic Word Size (LAWS) logic will lead to enormous reduction in processing speed of FHE algorithms by reducing the accumulation of the computation noise in lattice encryption, which translates into improved computation runtime by reducing the frequency and computational penalty of bootstrapping. DPRIVE will develop a LAWS accelerator to process native word sizes expected to be thousands of bits long, in contrast to conventional computing systems, which operate with physical word lengths no longer than 64-bits. Initial analysis of processing with LAWS indicates an improvement of four orders of magnitude in FHE algorithm processing speeds.

Building a LAWS accelerator will stress most aspects of chip design. The key technical challenges that will be overcome in DPRIVE include:

1. Design FHE hardware accelerator based on LAWS architecture that can be formally verified within minutes to hours. While many challenges exist in the design of a LAWS accelerator (e.g., routing, physical area, power management), circuit verification is among the most difficult. Formal verification is essential for two reasons:

a. Cryptographic circuits have a high burden of proof for mathematical correctness that elevates the need for full circuit verification; and

b. LAWS circuits are large and lead to a combinatorial explosion of the circuit state space. Previous verification attempts of large-word size multipliers timeout at word sizes of 256 bits. In contrast, DPRIVE seeks innovations in using symbolic arithmetic representation of the gate logic to significantly cut down on verification time. Such representation should for example reduce execution time for the verification of LAWS multipliers down to minutes for a 256-bit multiplier of reasonable sophistication and to only a few hours for full formal verification of multipliers in the thousands of bits.

2. LAWS memory management that provides ≥10x run-time speed-up with tolerable circuit area. Optimization efforts to parallelize data execution show 10 – 100x speed improvement. Implementing storage for parallel operations in LAWS processors has two big issues:

a. Moving large words around a processor to enable simultaneous execution on multiple words; and

b. Storing enough words in local caches on the processor to enable parallel data processing and efficient pipelining.

Today’s processors use input and output (I/O) models for memory fetches for around 512 bits (64 bytes) at a time, which means added latency of eight fetches per word in a 4k LAWS architecture. Furthermore, storing large words leads to very large local caches. A common processor today can have a 16 MB cache with 64-bit words, which results in storing 256k words at a time. Such caching consumes approximately 40 mm2 of the chip area in a 14 nm node size. An FHE processor with a cache that holds the same number of 4k words means a 1 GB cache and a chip size of 2,545 mm2, which is well beyond the reticle limits of chip manufacturing processes at the 14-12nm node size (i.e., just over 800 mm2). The memory challenge of the DPRIVE program is to remove memory fetch penalties for LAWS through new I/O designs while also improving the computational speeds of execution on memory through parallelized operations, resulting in at least a factor of 10 reduction in computation time without breaking size limits of the chip production process. Size limits will be based on available DARPA/MTO shuttle runs of approximately 150 mm2.

3. Flexible data structures and programming models to enable parameterization of FHE algorithms. FHE algorithms have parameters that affect security and execution times.

Building an FHE accelerator should enable the parameterization of the FHE algorithm to support different user and application needs. Trying to attain optimal performance might lead to static implementations. By contrast, DPRIVE is looking for solutions that provide flexibility of the parameter space. DPRIVE will look at models of computation that provide mapping from the mathematical operations to the underlying hardware without excessively impacting the overall execution speed of an FHE algorithm. This is largely accomplished by taking advantage of vectorized computing models that affect much of the parameter space in question in order to provide at least a 10x improvement in speed over non-vectorized approaches, given the memory model designs of the second challenge. In addition, the data structures and programming models should support diversity of FHE scheme that are based on Ring Learning With Errors (RLWE) in the support of lattice encryption. Such could be implemented through programmable chaining of core mathematical operations common to most RLWE schemes.

DPRIVE focus areas are illustrated in Figure 4.

Figure 4: DPRIVE focus areas

In addition, DPRIVE must address challenges in the co-design of FHE algorithms, hardware, and software. The implementation of an FHE accelerator must bring together mathematical experts in FHE with designers of complex, unconventional circuits to implement an effective FHE accelerator. Software developers must be included in the co-design process to implement the supporting software and firmware. The coordinated strengths of the proposing teams in FHE algorithm design and in VLSI circuit design will be important factors towards team selection.

Software is critical to the use and viability of any accelerator implementation. Application programming interfaces (APIs) and associated software and hardware infrastructures to ingest the encrypted wide word size and interface with host processing are essential. Without software that enables a user to efficiently use the FHE accelerator within the domain of the host processor, a hardware solution, despite its potential, will never be used. There are a few existing and open source libraries for FHE processing. Proposers are strongly encouraged to adopt and use one of the existing libraries and provide a plan for interacting with the developers of such libraries to provide interfaces and other enhancements that are judged necessary to interoperate with the proposed FHE coprocessor. Note that the development of wholly new FHE software libraries is not within scope of the DPRIVE program.

The DPRIVE program and the resulting accelerators and supporting infrastructure will address the above described challenges and establish the foundation for and an implementation of FHE computation with minimal run-time burden. This will enable FHE to provide data security during data computation, storage, and transit – resulting in security for data in all states. Ease of use of the DPRIVE FHE accelerator hardware will be essential towards widespread uptake of the outcomes from DPRIVE and overall acceptance of FHE among diverse technology communities of interest.

C. Program Structure

The DPRIVE program will be a 42-month program composed of three phases, a base Phase 1 and optional Phases 2 and 3 described in this section. It is expected that fewer performers will be funded to participate in Phases 2 and 3 of the program. Options may be exercised, at the Government’s sole discretion, based on technical progress measured against the metrics and milestones defined in this BAA and funding availability.

Phase 1 of the three-phase program is a 15-month effort to produce the core logic of the FHE accelerator design. Teams must conduct the trade space analysis to determine optimal word size, design, and emulate all building block circuits required for FHE computation (e.g., multiply, add, modulo, shifts, and data transforms) and run on a circuit emulator a logistic regression analysis on a data set(s) to be provided by the government team on or before program kickoff. The designs must show:

• Formal verification of all logic blocks with ≥ 90% coverage in less than 1 day.

• Execution of the logistic regression problem of 1024 data points in less than 10 ms.

Phase 2 is a 15-month effort to finish the design of the full FHE accelerator based off the Phase 1 building blocks along with the memory architecture. The Phase 2 exam is the execution of a 7-layer convolutional neural network (CNN) inference against the CIFAR-10 data set in less than 250 ms (within 100x of plaintext operation on a standard workstation). Other metrics of this phase are:

• Formal verification of all logic blocks with 100% coverage in less than 1 day.

• Chip area: ≤ 150 mm2.

Phase 3 is the final 12-month effort to build out a working and fully-usable FHE accelerator with built-in I/O and appropriate hardware/firmware interfacing to a CPU to ensure full software and programming functionality. The final exam will measure the following metrics:

• Execution of 7-layer CNN inference against CIFAR-10 data set: ≤ 25 ms (within 10x).

• Execution of 7-layer CNN training against CIFAR-10 data set: ≤ 10 hours (within 10x).

• Demonstrate flexibility in the algorithm parameters:

• Plaintext modulus: 2 – 1024.

• Ciphertext modulus: 215 – 2500

• Ring dimension: 512 – 16384.

DARPA will provide the data sets used for Phase 1 evaluation, as well as the CNN structure to be used in Phases 2 and 3 for use with the standard CIFAR-10 data.

The DPRIVE program will not entertain alternative computing models outside of silicon.

Fabrication is expected to be at leading node geometries and utilize DARPA/MTO managed shuttle runs with a designated foundry. Instructions detailing this process will be provided after contract award. Additional details about this process are provided in Section I.H below.

D. Technical Area

DPRIVE will have a single Technical Area. Each DPRIVE team will need to address all required DPRIVE development activities (hardware + software + algorithms). This will include: FHE design and requirements analysis, FHE accelerator design analysis and trade-offs, formal verification of the circuit design, algorithm-accelerator co-design development, building block and integrated accelerator design development, building block and full FHE accelerator design and performance verification, host processing environment development, FHE supporting accelerator software development, user interface and tools, accelerator implementation, accelerator fabrication, accelerator supporting board and host CPU interface implementation, and accelerator evaluation and demonstration.

As discussed above, DPRIVE’s single Technical Area will specifically include the identified key technical challenges:

Design FHE hardware accelerator based on LAWS architecture that can be formally verified within minutes to hours.

LAWS memory management that provides ≥10x run-time speed up with tolerable circuit area.

Flexible data structures and programming models to enable parameterization of FHE algorithms.

Proposals must not address cryptanalysis approaches or research. The proposed effort must specifically address only the development of an FHE accelerator, as described in this solicitation.

E. Schedule/Milestones

The DPRIVE program will have three phases over a total program length of 42 months. The overall objective is the implementation of a working FHE accelerator and supporting software and hardware infrastructure that enables the performance of FHE computation at runtimes within an order of magnitude of that of conventional unencrypted computation. The DPRIVE program schedule is shown in Figure 5 and the detailed objectives/outcomes of each phase are given in Figure 6.

Figure 5: DPRIVE Program Schedule

The DPRIVE program metrics will address the design maturity, formal design verification, and performance in terms of power dissipation and speed evaluated against the same data sets and

CNN models for all performers. The data sets will be provided at program kickoff. The CNN model is described under Section H. Government Furnished Equipment/Property/Information.

Program Metrics by phase are given in Figure 6.

Figure 6: DPRIVE program metrics. Specific details of the algorithms and datasets to be tested will be provided by the US Government once the DPRIVE program is underway. Metrics in parenthesis indicate expected runtime penalty vs. plaintext operations.

F. Deliverables

All performers shall deliver detailed plans at program kickoff and upon execution of subsequent option awards, quarterly technical reports, monthly financial reports including updated expenditures, and end of phase technical reports providing details on work and accomplishments performed during that program phase. Performers shall prepare and submit briefing materials and participate in quarterly progress reviews, either by teleconference or at the performer’s site, at the discretion of the DARPA program manager. All performers shall travel to and support bi-annual program reviews scheduled at the DARPA program manager’s discretion.

In addition to periodically scheduled deliverables, performers will deliver in-depth technical briefings as requested by DARPA. DARPA estimates that such requests will be made no more than once per quarter.

Upon the completion of each phase, the performer end-of-phase technical report must include:

A documented description of the technical development and achievements accomplished.

DPRIVE demonstrations and a documented description of capabilities developed and supported.

Details and description of the work performed, and capabilities developed and how the capabilities meet, exceed, or fall short of program goals as established in this BAA.

All design files corresponding to the deliverables, including high level descriptive language implementations of the circuit design, Verilog (or equivalent), GDSII layouts, etc.

Plans and projections for the next program phase, with DoD and commercial transition opportunities specifically provided upon the completion of all phases.

The deliverables by program phase are as follows:

Phase 1:

Initial design description and emulation of DPRIVE accelerator building blocks.

Initial FHE algorithm implementations in the emulated accelerator blocks. Descriptions of the operation and implementation of the FHE algorithms must be thoroughly documented.

Analysis of the emulated performance per subcomponent/design block.

Formal verification of all building blocks, with results documented and providing details of the approach to formal verification. The formal verification will not only validate the circuit design for accuracy, but will also provide a scalable and automatable proof of correctness that the circuit performance is formally verifiable under all circumstances.

Analysis of the proposed design, including performance/complexity trade-offs and detailed descriptions of proposed design decisions. This analysis should show where DPRIVE program metrics and goals were met. For situations where such goals were not met, provide compelling descriptions of alternative metrics.

Software code and configurations for the accelerator emulation of a logistic operation (see Figure 5).

A full Critical Design Review (CDR) of the proposed DPRIVE accelerator, providing the proposed accelerator design, anticipated performance (based on the accelerator emulation for a logistic regression problem), relevant implementation decisions, performance versus program metrics and goals, and supporting software environment. All high level design, register transfer logic (RTL) Verilog, netlist, design (e.g., computer aided design (CAD)) files, and code produced during Phase 1 shall be made available to the government.

Phase 2:

Further enhancements and documentation of the DPRIVE accelerator design description.

Complete DPRIVE accelerator tapeout ready for fabrication of the accelerator.

Emulation of test chip comprised of core ALU building blocks developed in Phase 1.

Full accelerator emulation based analysis of the accelerator design versus DPRIVE program metrics and goals, including performance speed and power consumption metrics. The emulation will include execution of FHE algorithms running an application on a full accelerator emulator (application as given in Figure 5).

Formal verification of the entire FHE chip, with results documented. Provide details of the approach to formal verification that not only validates the circuit design for accuracy, but provides a scalable and automatable proof of correctness that the circuit performance is formally verifiable under all circumstances.

DPRIVE accelerator package and supporting board design.

Provide initial evaluation through emulation of full DPRIVE chip using government provided data set.

Full chip DPRIVE accelerator design review, including review of the design tapeout, execution performance (including accelerator emulation results for a logistic regression and CNN inference), description of relevant implementation decisions, performance versus program metrics and goals, and supporting software environment. All high level design, RTL, Verilog, netlist, etc., CAD files, and code shall be made available to the government.

Phase 3:

Implemented and packaged DPRIVE accelerator on functional FHE processing board.

The functional DPRIVE FHE board will include I/O, interfaces, and firmware/software to support integration into a functional CPU computing system.

Full demonstration, verification, testing, and analysis of DPRIVE accelerator with full software stack and environment.

Documentation, user manuals, and example training material for using the FHE processing board.

Analysis of the design versus DPRIVE program metrics and goals, including performance speed and power consumption metrics.

Evaluate government provided data set through DPRIVE accelerator.

Full software environment implementation, with execution on the DPRIVE accelerator system and porting to the hosting CPU, including software driver from CPU to load data and execute FHE algorithms (as per Figure 5).

FHE algorithm demonstrations on the accelerator.

Evaluate performance of and validate accelerator.

Full chip DPRIVE accelerator design review, including review of DPRIVE accelerator implementation, performance of FHE accelerator system (including on the accelerator system for CNN training and inference), relevant implementation decisions, performance versus program metrics and goals, and supporting software environment. All high level design, RTL, Verilog, netlist, etc., CAD files, and code shall be made available to the government.

Prior to conclusion of Phase 3, the performer will deliver a minimum of three complete FHE prototype boards to one or more selected US Government agencies for their own internal evaluation and application exploration. Training material, engineering time, and documentation are expected as part of this deliverable.

Important Points:

1. DPRIVE calls for the implementation of fully homomorphic encryption in the accelerator to be delivered. “Somewhat homomorphic” (SHE), “partially homomorphic” (PHE), multi-party compute (MPC) and garbled circuit (GC), or other solutions/approaches to privacy protecting/preserving computation are not acceptable to the program in that they are mathematically incomplete in the underlying mathematical operations that must take place over the encrypted data, or otherwise do not address the goals of the program.

2. Field Programmable Gate Array (FPGA) or graphics processing unit (GPU) solutions are not of interest to the program. Multichip module approaches will be considered provided a clear path toward meeting program goals and objectives is proposed.

3. The underlying approach and algorithmic architectures will be based on lattice encryption.

Proposers are recommended to follow FHE approaches as proposed by the HomomorphicEncryption.org standards consortium, in particular as described in the document:

http://homomorphicencryption.org/wp-content/uploads/2018/11/HomomorphicEncryptionStandardv1.1.pdf

Equivalent approaches to FHE lattice encryption from other organizations will likewise be considered.

4. It is envisioned that a lambda value of 128 bits of security should be adequate for most applications, but that design flexibility should allow a range of values for lambda as provided by the plaintext modulus, ciphertext modulus, and ring size in the program metrics.

5. DPRIVE proposals should support FHE algorithms based on Ring Learning with Errors (RLWE). Flexible designs able to support two or three different FHE algorithms, but at a minimum must support BGV3, based on the mathematics of RLWE through reprogramming the processor will be reviewed favorably. All proposed solutions must be able to run the BGV FHE scheme. This is required in order to provide a means for performance comparison across performer implementations.

G. Commercialization

Proposers must provide a plan for commercialization, promoting a vision and associated commercialization steps at each phase within this program, as well as upon program completion.

To this end, proposers are also encouraged to leverage DARPA’s Embedded Entrepreneur Initiative (EEI), which provides guidance and additional funding towards the implementation of a commercialization plan for technologies developed under DARPA Microsystems Technology Office sponsorship (see Section IV Application and Submission Information / Paragraph J under “Section II. Detailed Proposal Information”). Supporting activities towards commercialization may include participation in industry standards development, publications, conference demonstration, academic design challenge, and industry association engagement. It is anticipated that the DPRIVE program will influence Third Generation Partnership Project (3GPP) and Internet Engineering Task Force (IETF) standards. DPRIVE program members are encouraged to coordinate with the Office of the Undersecretary of Defense for Research and Engineering (USD(R&E)) and/or the Office of Science and Technology Policy (OSTP) on DoD standards, and initiate interactions with 3GPP, IETF, and other data communications standardization bodies.

3 Z. Brakerski, C. Gentry, and V. Vaikuntanathan. Fully Homomorphic Encryption without Bootstrapping, In ITCS 2012.

http://homomorphicencryption.org/wp-content/uploads/2018/11/HomomorphicEncryptionStandardv1.1.pdf http://homomorphicencryption.org/wp-content/uploads/2018/11/HomomorphicEncryptionStandardv1.1.pdf

H. Government Furnished Equipment/Property/Information

DPRIVE accelerator implementation is expected to use available DARPA/MTO device fabrication access. This will be accomplished by performers signing up for accounts/access to the government designated fabrication process. DPRIVE accelerator designs will be fabricated as riders on shared DARPA/MTO wafer fabrication runs. Details on shared fabrication runs will be provided to proposers selected as performers. The fabrication costs for DARPA/MTO shared fabrication runs will be borne by the government.

The maximum size of fabrication rider designs is anticipated to be on the order of 150 mm2.

Proposers should consider this design limitation in DPRIVE proposals and plan to develop DPRIVE accelerator designs that utilize MTO fabrication access and are within 150 mm2.

A design larger than 150 mm2 can be proposed, but may not exceed the reticle limit of the designated fabrication process (typically around 800 mm2). If a proposer, for performance or functional reasons, submits a proposal with a design larger than 150 mm2 they will need to include in their technical and cost proposals a full description of the rational for the design size being proposed and the associated cost and funding approach. Specifically, this should include a detailed explanation of the performance improvement and impact to program metrics. In order to support a design greater than 150 mm2, the proposer will not be able to utilize MTO fabrication access and will, therefore, have to include the full cost of fabrication for their DPRIVE accelerator in the cost proposal. When the full cost of fabrication is included in the cost proposal, the proposer offsetting the increased cost (the anticipated delta fabrication cost associated with a design greater than 150 mm2) by contributing cost share is encouraged.

The DPRIVE program will work with the Air Force Research Laboratory (AFRL) to provide access to and use of AFRL’s Palladium tools and installation by DPRIVE performers to support design simulation and emulation. If a performer plans to use these government provided resources, they should provide an estimate in their proposal of their expected access requirements, including an anticipated schedule for use of these resources and the number of expected hours/runs using these resources. Use of the AFRL Palladium tools and installation is not required.

All data sets used in the program for the logistic regression and CNN training and inferences stages will be based on publically available data sets. The intention is not to advance the state of the art in unencrypted regression and CNN operations but to demonstrate the feasibility of performing such operations at highly accelerated speeds on a FHE coprocessor. It is for this reason that the DPRIVE program has selected data sets that are frequently encountered as benchmarks for a variety of approaches instead of classification problems.

CNN Inference and Training

The CNN operations will make use of the CIFAR-10 data set4. This data set contains 60,000 32x32x3 color images assigned to 10 classes, with 6,000 images per class. A total of 50,000 training images and 10,000 test images are available in the set.

To assure consistency in the evaluation of the CNN performance across teams, it is required that the classification CNN of Figure 7 be implemented for unencrypted (baseline) operations and the FHE operations.

1) Convolution: input image 3 × 32 × 32, window size 3 × 3, stride (1, 1), pad (1, 1), number of output channels 64: R64×1024 ← R64×27· R27×1024.

2) ReLU Activation: calculates ReLU for each input.

3) Convolution: window size 3 × 3, stride (1, 1), pad (1, 1), number of output channels 64: R64×1024 ← R64×576· R576×1024.

4) ReLU Activation: calculates ReLU for each input.

5) Mean Pooling: window size 1 × 2 × 2, outputs R64×16×16.

6) Convolution: window size 3 × 3, stride (1, 1), pad (1, 1), number of output channels 64: R64×256 ← R64×576· R576×256.

7) ReLU Activation: calculates ReLU for each input.

8) Convolution: window size 3 × 3, stride (1, 1), pad (1, 1), number of output channels 64: R64×256 ← R64×576· R576×256.

9) ReLU Activation: calculates ReLU for each input.

10) Mean Pooling: window size 1 × 2 × 2, outputs R64×16×16.

11) Convolution: window size 3 × 3, stride (1, 1), pad (1, 1), number of output channels 64: R64×64 ← R64×576· R576×64.

12) ReLU Activation: calculates ReLU for each input.

13) Convolution: window size 1 × 1, stride (1, 1), number of output channels of 64: R64×64 ← R64×64· R64×64.

14) ReLU Activation: calculates ReLU for each input.

15) Convolution: window size 1 × 1, stride (1, 1), number of output channels of 16: R16×64 ← R16×64·R64×64.

16) ReLU Activation: calculates ReLU for each input.

17) Fully Connected Layer: fully connects the incoming 1024 nodes to the outgoing 10 nodes: R10×1 ← R10×1024· R1024×1.

Figure 7. Convolution Neural Network for CIFAR-10 dataset. The network comprises 6 inner convolution-activation layers with a final fully connected layer5. R means the space of real numbers.

I. Intellectual Property

Any proposed use of intellectual property (patents, proprietary information, etc.) should be clearly identified in the proposal. Identify all intellectual property claims to future results, prototypes, and deliverables. Explain how these claims may limit the Government use of the technology developed under the DPRIVE program or development of derivative technologies.

For forms to be completed regarding intellectual property, see Section IV.B.10. If there are no intellectual proprietary claims, this should be stated.

4 Learning Multiple Layers of Features from Tiny Images Alex Krizhevsky April 8, 2009 https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf 5 Jian Liu, Mika Juuti, Yao Lu, and N. Asokan. Oblivious neural network predictions via minionn transformations.

In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS 2017, Dallas, TX, USA, October 30 - November 03, 2017, pages 619–631, 2017.

DARPA expects application-level software and frameworks developed under the DPRIVE program to be licensed and executed as free and open source software (FOSS) using a standard model for open source licensing, management, and maintenance practices. FOSS for FHE will continue to foster the research and development of FHE practices for the wider adoption among network and mobile phone technology sectors.

II. Award Information

A. General Award Information

Multiple awards are anticipated. The amount of resources made available under this BAA will depend on the quality of the proposals received and the availability of funds.

The Government reserves the right to select for negotiation all, some, one, or none of the proposals received in response to this solicitation, and to make awards without discussions with proposers. The Government also reserves the right to conduct discussions if it is later determined to be necessary. If warranted, portions of resulting awards may be segregated into pre-priced options. Additionally, DARPA reserves the right to accept proposals in their entirety or to select only portions of proposals for award. In the event that DARPA desires to award only portions of a proposal, negotiations may be opened with that proposer. The Government reserves the right to fund proposals in phases with options for continued work at the end of one or more of the phases, as applicable.

Awards under this BAA will be made to proposers on the basis of the evaluation criteria listed below (see section labeled “Application Review Information,” Sec. V.), and program balance to provide overall value to the Government. The Government reserves the right to request any additional, necessary documentation once it makes the award instrument determination. Such additional information may include but is not limited to Representations and Certifications (see Section VI.B.4., “Representations and Certifications”). The Government reserves the right to remove proposers from award consideration should the parties fail to reach agreement on award terms, conditions and cost/price within a reasonable time or the proposer fails to timely provide requested additional information. Proposals identified for negotiation may result in a procurement contract, grant, cooperative agreement, or other transaction, depending upon the nature of the work proposed, the required degree of interaction between parties, whether or not the research is classified as Fundamental Research, and other factors.

Proposers looking for innovative, commercial-like contractual arrangements are encouraged to consider requesting Other Transactions. To understand the flexibility and options associated with Other Transactions, consult http://www.darpa.mil/work-with-us/contract-management#OtherTransactions.

In accordance with 10 U.S.C. § 2371b(f), the Government may award a follow-on production contract or Other Transaction (OT) for any OT awarded under this BAA if: (1) that participant in the OT, or a recognized successor in interest to the OT, successfully completed the entire prototype project provided for in the OT, as modified; and (2) the OT provides for the award of a http://www.darpa.mil/work-with-us/contract-management#OtherTransactions http://www.darpa.mil/work-with-us/contract-management#OtherTransactions follow-on production contract or OT to the participant, or a recognized successor in interest to the OT.

In all cases, the Government contracting officer shall have sole discretion to select award instrument type, regardless of instrument type proposed, and to negotiate all instrument terms and conditions with selectees. DARPA will apply publication or other restrictions, as necessary, if it determines that the research resulting from the proposed effort will present a high likelihood of disclosing performance characteristics of military systems or manufacturing technologies that are unique and critical to defense. Any award resulting from such a determination will include a requirement for DARPA permission before publishing any information or results on the program. For more information on publication restrictions, see the section below on Fundamental Research.

For the effort solicited under this BAA, OT agreements are encouraged for non-traditional teams and those proposing research with potential commercial application. As a resource, the Model OT Agreement for ERI programs has been provided on the DARPA Acquisition Innovation website, under the title “Microsystems Technology Office (MTO) Electronics Resurgence Initiative (ERI)” (see https://acquisitioninnovation.darpa.mil/samples-and-resources/darpa-ot-programs).

B. Fundamental Research

It is DoD policy that the publication of products of fundamental research will remain unrestricted to the maximum extent possible. National Security Decision Directive (NSDD) 189 defines fundamental research as follows:

‘Fundamental research’ means basic and applied research in science and engineering, the results of which ordinarily are published and shared broadly within the scientific community, as distinguished from proprietary research and from industrial development, design, production, and product utilization, the results of which ordinarily are restricted for proprietary or national security reasons.

As of the date of publication of this BAA, the Government expects that program goals as described herein may be met by proposed efforts for…