DARPA-BAA-16-56_ReImagine_Amendment2_14Nov2016.pdf

Broad Agency Announcement Reconfigurable Imaging (ReImagine)

Microsystems Technology Office

DARPA-BAA-16-56

SEPTEMBER 19, 2016

(Amendment No. 02: As amended through November 14, 2016)

Table of Contents

PART I: OVERVIEW INFORMATION

A. Introduction and Background B. Program Objective

1. Technical Area 1 (TA1): Single or multi-color passive imager architecture and algorithms

2. Technical Area 2 (TA2): Hybrid active/passive imager architecture and algorithms 7

3. Technical Area 3 (TA3): Innovative concepts for imaging systems with internal feedback C. Program Structure, Milestones, Schedule, and Deliverables

1. Technical Areas 1 and 2

2. Technical Area 3

II. Award Information III. Eligibility Information

A. Eligible Applicants B. Procurement Integrity, Standards of Conduct, Ethical Considerations, and

Organizational Conflicts of Interest C. Cost Sharing/Matching D. Other Eligibility Criteria

1. Collaborative Efforts IV. Application and Submission Information

A. Address to Request Application Package B. Content and Form of Application Submission

1. Security Information

2. Proprietary Information

3. Full Proposal Format

4. Proposal Submission Information

5. Submission Dates and Times

6. Funding Restrictions

7. Other Submission Requirements

V. Application Review Information A. Evaluation Criteria B. Review and Selection Process

VI. Award Administration Information A. Selection Notices B. Administrative and National Policy Requirements

1. Meeting and Travel Requirements

2. Human Subjects Research

3. Animal Use

4. Export Control

5. Subcontracting

6. Electronic and Information Technology

7. Employment Eligibility Verification

8. Reserved

9. System for Award Management (SAM) and Universal Identifier Requirements

10. Reporting Executive Compensation and First-Tier Subcontract Awards

11. Updates of Information Regarding Responsibility Matters

12. Representations by Corporations Regarding an Unpaid Delinquent Tax Liability or a Felony Conviction under any Federal Law

13. Cost Accounting Standards (CAS) Notices and Certification

14. Controlled Unclassified Information (CUI) on Non-DoD Information Systems

15. Safeguarding of Covered Defense Information and Cyber Incident Reporting

16. Prohibition on Contracting with Entities that Require Certain Internal

Confidentiality Agreements C. Reporting D. Electronic Systems

1. Representations and Certifications

2. Wide Area Work Flow (WAWF)

3. i-Edison

VII. Agency Contacts VIII. Other Information

A. Intellectual Property Procurement Contract Proposers

1. Noncommercial Items (Technical Data and Computer Software)

2. Commercial Items (Technical Data and Computer Software)

B. Non-Procurement Contract Proposers – Noncommercial and Commercial Items (Technical Data and Computer Software)

C. All Proposers – Patents D. All Proposers – Intellectual Property Representations E. Other Transactions (OTs):

ATTACHMENT 1: Cost Volume Proposer Checklist ATTACHMENT 2: Proposal Summary Slide Template

PART I: OVERVIEW INFORMATION

Federal Agency Name – Defense Advanced Research Projects Agency (DARPA), Microsystems Technology Office (MTO)

Funding Opportunity Title – Reconfigurable Imaging (ReImagine) Announcement Type – Initial Announcement Funding Opportunity Number – DARPA-BAA-16-56 Catalog of Federal Domestic Assistance Numbers (CFDA) – 12.910 Research and

Technology Development Dates o Posting Date: September 19, 2016 o Proposal Due Date: November 16, 2016 o Estimated period of performance start date: 1 April 2017

Concise description of the funding opportunity: DARPA is soliciting research proposals to demonstrate multi-functional imaging sensors that are reconfigurable through software. Proposers will build around a common digital framework that can be customized for specific applications. Both passive and active modes are desired. Also of interest are proposals that develop adaptive algorithms that optimize the operation of a reconfigurable sensor in real time to optimize information collection.

Total amount of money available to be awarded: Approximately $20M in awards are anticipated.

Anticipated individual awards – Multiple awards are anticipated.

Anticipated funding type - 6.2 and/or 6.3 Types of instruments that may be awarded – Procurement contract, grant (TA3 Only), cooperative agreement (TA3 Only) or other transaction Any cost sharing requirements – None Agency contact o Dr. Jay Lewis, Program Manager BAA Coordinator: DARPA-BAA-16-56@darpa.mil

DARPA/MTO

ATTN: DARPA-BAA-16-56

675 North Randolph Street Arlington, VA 22203-2114

FAX: 703-248-8008

EMAIL: DARPA-BAA-16-56@darpa.mil

PROPOSERS ARE CAUTIONED THAT EVALUATION RATINGS MAY BE LOWERED AND/OR

PROPOSALS REJECTED IF PROPOSAL PREPARATION (PROPOSAL FORMAT, CONTENT, ETC.) AND/OR

SUBMITTAL INSTRUCTIONS ARE NOT FOLLOWED.

THOSE INTENDING TO SUBMIT A PROPOSAL FOR AN ASSISTANCE INSTRUMENT (GRANT OR

COOPERATIVE AGREEMENT) ARE STRONGLY ENCOURAGED TO READ THE INSTRUCTIONS

PROVIDED AT SECTION IV(B)(4) REGARDING THE TIME REQUIRED TO RECEIVE VALIDATION OF

SUBMISSIONS MADE THROUGH GRANTS.GOV. PROPOSALS THAT ARE VALIDATED AFTER THE

PROPOSAL DUE DATE/TIME WILL BE CONSIDERED LATE AND, AS SUCH, WILL NOT BE REVIEWED.

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

The Defense Advanced Research Projects Agency (DARPA) often selects its research efforts through the Broad Agency Announcement (BAA) process. This BAA is being issued, and any resultant selection will be made, using the procedures under Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016, and 2 C.F.R. § 200, as applicable. Any negotiations and/or awards will use procedures under FAR 15.4 (Contract Pricing) or 2 CFR 200, Subpart E (Cost Principles), as applicable. Proposals received as a result of this BAA shall be evaluated in accordance with evaluation criteria specified herein through a scientific review process.

DARPA BAAs are posted on the Federal Business Opportunities (FedBizOpps) website, http://www.fbo.gov/, and, as applicable, the Grants.gov website at http://www.grants.gov/. The following information is for those wishing to respond to the BAA.

DARPA is soliciting innovative research proposals to develop concepts and demonstrate an architecture for a software-reconfigurable, multi-mode imaging system. The resulting camera technology will incorporate functions that are normally not accessible within a single focal plane array (FPA) by configuring regions-of-interest (ROIs) that operate independently of other regions of the array, and by reconfiguring the measurements being made in the imaging array in response to the scene. Proposed research should investigate innovative approaches that enable revolutionary advances in science, devices, or systems. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice.

A. Introduction and Background

The objectives for most camera designers include maximizing spatial resolution and signal-to-noise (SNR). Yet a wealth of information in the optical domain is lost under these constraints.

Specialty cameras exist to capture other types of information, for example in the frequency domain, the spectral domain, or the resolution of depth. But these cameras are not normally able to provide high SNR imagery at high spatial resolution from a single focal plane, and are used relatively infrequently due to the system demands of adding camera systems. Today’s imaging systems primarily perform only a single or limited set of measurements due in part to the underlying readout integrated circuits (ROICs), which sample the signal of interest and transfer these values off of the chip. ROICs are typically designed for a very specific mode of operation, and in essence are application specific integrated circuits (ASICs).

An imaging system that autonomously extracts the most relevant information, using a single sensor, and based only on the context in the scene would revolutionize a wide variety of military and commercial applications. This requires the development of a software-configurable array that enables simultaneous and distinct imaging modes in different ROIs. This would provide capabilities that previously required multiple sensors. It also requires algorithms that adapt the sensor configuration in real time based on context, and creates a consistent marketplace for information that seeks to maximize the value of making one measurement relative to the cost of missing others.

Over the last decade, the emergence of imaging arrays with in-pixel analog-to-digital conversion (ADC) has enabled innovative concepts for FPAs with wide dynamic range and in-pixel processing1. Similar pixel architectures have been used for high performance light detection and ranging (LIDAR) measurements with both framed and asynchronous operation. However, pixel pitches for arrays that both digitize and accumulate signals in the pixel remain at 20 µm or larger, and these designs are typically fixed-logic ASICs. Using an advanced node complementary metal-oxide semiconductor (CMOS) process provides an opportunity to both reduce pixel pitch and also insert sufficient programmable logic to enable a software definable platform. In addition, separating the analog components that interface with the detector into a separate layer with per-pixel interconnects introduces the ability to customize an application agnostic all-digital layer for a wide range of applications.

B. Program Objective

The objectives of the ReImagine program are to demonstrate that a software-reconfigurable imaging system can enable revolutionary capabilities, present a new approach to application development that is more similar to field programmable gate array (FPGA) based design than to ASIC design, and to develop the underlying theory and algorithms that learn to collect the most valuable information when the sensor can be configured for a variety of measurements. The ReImagine program aims to demonstrate that a single ROIC architecture can be configured to accommodate multiple modes of imaging operations that may be defined after the chip has been designed. With the use of 3-D integration, it will be possible to customize the sensor to interface with virtually any type of imaging sensor (e.g. photodiode, photoconductor, avalanche photodiode, or bolometer) and to optimize it for any spectral band (e.g. ultraviolet (UV) through very long-wave infrared (VLWIR)). More importantly, it will be possible to adapt the mode of operation either through manual user control, through preset routines that can change many times per second, or in response to context derived from the scene being observed. For example, a single imager could present simultaneous ROIs that can run at high resolution (i.e. foveated imaging), or at high frame rate. ReImagine ROICs will also demonstrate that efficient computation within an ROI can enable real-time analysis on much more complex scenes than traditional systems. ReImagine will build on this architecture to develop a concept of operation, the application requirements, the modes of operation, and the algorithms that will be used. The result will be more actionable information to the warfighter (or the warfighter’s automatic response system) than has ever been possible from a single imaging sensor.

In addition to multiple passive imaging functions, the ability to incorporate range detection into a high resolution, low noise imaging system offers a potential revolutionary capability. LIDAR systems today are predominantly scanning devices that contain large moving components and do not provide high quality context imagery. 2-D imaging LIDAR systems have been demonstrated and are able to acquire 3-D imagery in framing or asynchronous modes. Both direct detect and coherent receiver arrays have been demonstrated, each with distinct advantages for different applications. However, in all cases, high data rates limit the spatial resolution of the sensor, and the demonstration of both passive imaging and active LIDAR modes in a large (> 1 MPixel)

1 K.I. Schulz et al., “Digital-Pixel Focal Plane Array Technology,” Lincoln Laboratory Journal, vol. 20 (2), 36-51 (2014).

array has not been demonstrated. A ReImagine dual-mode sensor would provide the ability to collect high data rate LIDAR measurements within a configurable ROI, while continuing to measure passive context imagery.

DARPA seeks innovative proposals in the following Technical Areas. Proposers may propose to more than one technical area, but must fulfill all the requirements defined for each, the tasks and costs associated with each Technical Area must be clearly delineated and easily separable, and any technical and/or cost interdependencies clearly identified (and minimized to the maximum extent possible):

1. Technical Area 1 (TA1): Single or multi-color passive imager architecture and algorithms

TA1 aims to design and develop a single or multi-color passive camera architecture and supporting algorithms, which can support a variety of technical objectives that are not currently possible from a single FPA. Spectral bands of interest span from UV to the very long-wave infrared (VLWIR), or wavelengths approximately 0.25 µm – 14 µm, and should be driven by the proposed application. Multi-color imagers may be designed to integrate the signal from different spectral bands either simultaneously or consecutively.

2. Technical Area 2 (TA2): Hybrid active/passive imager architecture and algorithms

TA2 aims to design and develop a hybrid active/passive imager architecture, where passive mode operation is based on traditional intensity measurements across an image array, and active mode is based on time-of-flight (TOF) measurements for 3-D range information (e.g. LIDAR mode).

Moreover, the array can be configured to perform active mode measurements in specific ROIs, while simultaneously operating in passive mode in the remainder of the array. While TA2 efforts should demonstrate 3-D mode operation with integrated laser sub-systems, TA2 proposals should leverage existing laser sources and pointing systems. Proposals for other types of active mode imaging are of interest and should apply to TA2.

3. Technical Area 3 (TA3): Innovative concepts for imaging systems with internal feedback

TA3 will explore adaptive algorithms for reconfigurable imaging systems. The flow of information in today’s imaging systems is exclusively from the sensor to image processing and/or the user, and object, gesture, or activity recognition algorithms use data with parameters that do not change over time. The ReImagine architecture endeavors to provide an imaging system that can change the nature of data being measured, either spatially, temporally, or spectrally; either as intensity or time; and either frame-, change-, or event-driven. TA3 proposals should explore new concepts in active learning that can determine the type of data that should be collected, both as a function of location and time. The algorithms should maximize information content and enable decisions, based on the context of the scene and the predicted value of various types of data.

C. Program Structure, Milestones, Schedule, and Deliverables

1. Technical Areas 1 and 2

Use of either a procurement contract or other transaction (OT) award instrument is permitted for TA1 and TA2.

The central objective of TA1 and TA2 is to use a Government-furnished reconfigurable IC and development platform to experimentally demonstrate an imaging architecture in which reconfigurability provides a revolutionary capability. ReImagine proposals should consider revolutionary designs and applications previously unattainable from a single FPA by exploiting the reconfigurable functionality of the envisioned FPA architecture. Proposals suggesting incremental improvements to state of the art ROIC technologies are discouraged.

Figure 1 shows a notional drawing of the 3-layer ReImagine architecture. The foundation for ReImagine will be the Government-provided common digital hardware layer, Tier 1, with associated software for configuring the chip. While ReImagine performers will provide input to the design process for the common digital layer, proposals to develop the Tier 1 layer are not of interest. Common digital ROICs with per-pixel ADCs can be considered to have five primary components, as depicted in Figure 1. Each item is described briefly below:

1. The detector, where each pixelated detector is connected to the circuitry below it;

2. Mixed signal front end circuitry, in which analog signals from the photodetector are converted to digital pulses;

3. The digital registers that are typically used to count or measure the timing of pulses that represent photocurrent packets;

4. Pixel level signal processing and routing that is used for basic computation and routing of data out of the array;

5. Peripheral processing and multiplexing, which can include general or specific signal processing and logic, as well as multiplexing and input/output (I/O) resources.

In the ReImagine configuration, the detectors reside in Tier 3. Proposals to develop novel detector technology are not of interest. The mixed signal layer resides in Tier 2. The separation of the mixed signal and digital components into separate layers enables a common digital architecture to be adapted for a variety of photodetector technologies using a custom Tier 2 design. It also enables the use of the optimum CMOS feature size and voltage for the analog components. Components 3 – 5 from the list above will reside in Tier 1. Note that Figure 1 is notional and other configurations that achieve the same functionality are acceptable. However this BAA will used terminology consistent with Figure 1 to describe the functions of the imaging architecture.

Figure 1. Illustration showing the three layers of the notional ReImagine architecture and the baseline circuit functionality in each layer. The circled numbers refer to the components described in the text above.

TA1 and TA2 performers should design and fabricate Tiers 2 and 3, integrate them with the Government-provided Tier 1, and develop any necessary firmware to operate the resulting camera. In contrast to Tier 1, the front end analog layer (Tier 2) and the detector layer (Tier 3) are expected to be application dependent. Tier 2 and Tier 3 should contain arrays of sensors and front end circuitry, respectively, with one-to-one correspondence. Tier 2 will provide CMOS-compatible digital pulses that may be in response to photocurrent or other signal triggers. Tier 3 will consist of a detector layer suitable for the operational purpose of the imager.

As part of the ReImagine program, the Government plans to develop and provide two generations of reconfigurable integrated circuits (ICs) (Gen-1 and Gen-2) and their associated configuration software development tools, as Government Furnished Property (GFP). A detailed list and approximate dates for transfer of GFP are provided below (See “Government Furnished Property/Information”).

The resources in digital ROICs with per-pixel ADCs have historically been defined at the time of chip design to include an analog front end that serves as an interface between the detector and the digital circuitry and a fixed number of digital registers. The number of counters and the bit depth per pixel are limited by the feature size of the CMOS process and the detector pitch.

Control signals for these digital resources are typically global and shared by all pixels. The reconfigurable ICs that will be developed under this program seek to break this paradigm of fixed resources per pixel and instead provide banks of digital logic resources and reconfigurable routing channels that can be allocated to pixels, as needed, using the same programming techniques as FPGAs.

Signal inputs from the detector array to the Gen-1 IC will be composed of 4x4 arrays of digital I/O pins addressable by Tier 2, with reconfigurable routing channels to an array of (32) 8-bit registers. The registers can be independently configured to perform operations that include count up/down, timestamp, parallel orthogonal shift, or serial shift left/right. This level of configuration will allow for connectivity to pixel arrays of varying pitch, and enable the number of counters per detector pixel to be dynamically selectable. This reconfigurability will also provide more per-pixel resources when they are aggregated into larger pixels. Proposals should note where a pixel pitch larger than 10 µm will be demonstrated, and whether this is due to a constraint in the optics or detectors, or whether it is to make use of more per-pixel resources. Reconfigurable routing resources will also be provided for the distribution of control signals, and for moving data through the array. Surrounding the array of pixel circuitry will be peripheral banks of memory, digital signal processing (DSP) blocks, and reconfigurable logic to enable output formatting and on-chip processing. Proposers should assume that the Gen-1 ROIC will have a high speed I/O interface consisting of multiple output channels with 1-2 Gbps bandwidth per channel, for a total chip output bandwidth >= 16 Gbps. The Gen-2 bandwidth will be based on application requirements. For Gen-2, proposers should specify what bandwidths will be required for the high speed interface for their application.

Table 1 includes a description of the Gen-1 common digital layer hardware that should be assumed for ReImagine proposals, noting that the actual parameters may differ. It should be assumed that the digital layer will be fabricated using a commercial 14 nm CMOS process.

Proposals should explicitly state where additional features or resources would be needed to implement their proposed modes of operation, and whether those resources may increase the pixel pitch and/or power consumption. Note that Table 1 specifies a minimum interconnect pitch of 5 µm, and a pixel pitch of ≥ 10 µm. This is not meant to imply that there are ≥ 4 interconnects per pixel. This simply accounts for signal, digital I/O, power, and/or ground interconnects that may be necessary, and thus ensures that the proposed 3-D integration strategy is compatible with the actual floorplan of Tier 1.

Table 1. Notional characteristics for the Gen-1 Tier 1 IC.

Criterion Gen 1 Pixel Format 640 x 512 Pixel Pitch ≥ 10 µm Operating Temperature

77 – 300 K

Minimum Tier 1-2 Interconnect Pitch

5 µm

Digital Registers/Pixel

≥ 16 bits

Pixel Configuration 8 bits per counter, independently configurable o Count up/down or timing

Serial shift left/right Orthogonal data shift/route Resource configuration per 4x4 pixel sub-array

Tier 2 Interface 1 bidirectional digital I/O per 4x4 pixel sub-array

Routing 4 reconfigurable routing channels per 4x4 pixel sub-array

Peripheral Logic ≥ 10k look up tables ≥ 64 DSP blocks ≥ 1 MB memory

It is anticipated that the Gen-2 IC will expand on Gen-1 concepts to move towards an architecture that resembles an array of distributed processors, and may include more complex digital blocks similar to those surrounding the array in Gen-1 (e.g. memory, adders, multipliers, etc.). As discussed below, the Gen 2 design will ultimately be defined by incorporating input from TA1 and TA2 performers.

These reconfigurable ICs will be programmed though a combination of Government-furnished and open source FPGA computer-aided design (CAD) tools. Users will write a Verilog description of their desired ROIC configuration. Combining this Verilog description with provided Verilog models of the ReImagine digital resources, users will simulate their design to perform functional verification. Users will iterate through synthesis, optimization, and place and route implementation steps. Government-furnished software will generate a bitstream to program the IC through a standard Joint Test Action Group (JTAG) interface.

TA1 and TA2 proposals should place an emphasis on developing novel operating modes and algorithms which can significantly impact the application of interest. While TA3, discussed in more detail later, will focus on new control theory based on techniques that actively learn and adapt, control algorithms that enable appropriate mode-switching for TA1 and TA2 applications are within scope and should be included in TA1 and TA2 efforts. Successful proposals will:

Clearly describe an application, notional system-level platform, and mission of interest;

Quantitatively demonstrate the benefits of the ReImagine architecture to the application and mission;

Describe the proposed modes of operation in the context of pixel-level, FPA-level, and system-level operations;

Describe the principles or algorithms that control the modes of operation at a given time and location;

Provide a detailed description of the pixel-level design and features in Tier 2;

Provide a detailed description of the integration strategy, the prototype, and the final imaging demonstrations and test/validation strategy;

Define the challenges and associated risk mitigation strategies.

The ability to decouple the analog interface from the digital tier provides an opportunity for innovative or multi-mode front end designs. However, the objectives for the interface layer should be driven by the application requirements. Proposals to develop front end designs with lower noise and smaller values for the least significant bit (LSB) than the state of the art are of interest when this is relevant to the application, for example in low photon flux applications.

TA1 and TA2 efforts will have a three phase period of performance with a total duration of 45 months. Both TAs will have a 9-month Phase 0 base period, a 9-month Phase 1 Option, a 9-month Extended Phase 1 Option, and an 18-month Phase 2 Option, subject to the availability of funds and technical progress during the preceding phase(s). A schedule including all of the phases for each Technical Area is shown in Figure 2.

Figure 2. Program timeline for each phase of TA1-3.

Phase 0 – Base Period In Phase 0, performers will be provided with Gen-1 programming tools and associated documentation. These will should be used during Phase 0 to verify the operation and reconfigurability of the Tier 1 processor running the proposer-developed configuration. The CAD flow software and documentation will be provided as a combination of GFE and open source tools.

The overarching objectives of Phase 0 are to, a) develop and demonstrate the proposed ROIC configurations in software, b) define the operating parameters and requirements for a reconfigurable imaging system in the framework of the proposed application, c) identify shortcomings or desirable modifications to the Gen-1 Tier 1 design, and provide this as input for the Gen-2 design, and d) refine the approach and detailed plan for a Phase 1 demonstration.

Proposals should describe the activities that will be performed during Phase 0. Examples include elucidating the modes of operation specific to applications of interest, refining the detailed requirements for the digital layer, developing a framework for mode control, quantifying the impact of the multi-mode imager on the application of interest through simulation, and developing a detailed plan for development of a Phase 1 and/or Phase 2 demonstration camera.

Preliminary concepts, though not a full design, for the Tier 2 interface layer should be developed, and co-simulation of Tier 2 with the Tier 1 model is encouraged. Proposers should also develop a

Phase 0TA1/TA2

Phase 1 Design

Phase 1 Demonstration

Phase 2 Demonstration

Ph1 Design Opt.

18 Months9 Months

9 Months

18 Months

9 Months

Year 1 Year 2 Year 3 Year 4

45 Month Period of Performance

TA3 Phase 1 12 months

Phase 2 12 months

Phase 3 12 months

Kickoff

Kickoff power consumption framework to evaluate the system-level tradeoffs between computation within the array, on the periphery of the ROIC, and off-board for the application of interest. This should be done within a framework of total system power that accounts for required detector cooling, and include a preliminary thermal analysis of the 3-D ROIC.

Phase 1 – Option and Extended Option There are two parallel activities anticipated during Phase 1 for both TA1 and TA2: Phase 1 Demonstration and Phase 1 Design. Whether proposers should address one activity or both is discussed below. The parallel efforts should be clearly distinguishable and separable both in terms of technical scope and cost.

Phase I Demonstrations The objective of the 18 month Phase 1 Demonstration task is to demonstrate multi-functional imaging based on the Gen-1 reconfigurable IC. It is anticipated that Phase 1 efforts will deliver a multi-function prototype imager including a demonstration in a laboratory environment at the end of Phase 1. The Phase 1 demonstration must clearly show programmable functionality, unprecedented capability, and a path toward implementing further improvements in the Phase 2 demonstration. If proposers believe that the Gen-1 ROIC will not support their application, but that a Gen-2 ROIC could, they may elect not to propose to the Phase 1 Demonstration task, but this rationale should be clearly explained.

Proposals should provide a description of the following:

Application and principles of operation. This is the heart of ReImagine proposals.

Proposals should clearly describe innovative multi-functional modes of operation, embedded algorithms and/or autonomous control of functionality, and compare each mode of operation with the state of the art for a dedicated FPA designed specifically for that mode. These should be put in the context of relevant applications of interest to the DARPA mission. Priority should be given to concepts that demonstrate unprecedented capabilities based on reconfigurability.

Tier 2 design. The fundamental function of the Tier 2 layer is to convert sensor data into a digital pulse stream. An independent Tier 2 layer is an opportunity to customize the 3-D ROIC for the relevant detector technology and application, and to integrate novel designs not typically used in current digital ROIC technologies. For example, TA2 proposers may require multi-function capability for dual active/passive operation, and may leverage pixel-level control lines from Tier 1 to Tier 2. Proposals should describe the operation and design strategy for Tier 2. This should include a preliminary estimate of power consumption in representative conditions for Tier 2 and the anticipated CMOS production node.

3-D integration. Phase 1 demonstrations are expected to include pixel-level integration of the Tier 1 and Tier 2 layers, and one or more interconnects per pixel. Tier 1 die should be assumed to be available in wafer form and will be provided as GFE. Demonstrations will also require hybridization of the Tier 3 detector layer with the ROIC layers. A technical approach and rationale for both processes should be provided. The 3-D integration scheme should be repeatable and able to meet the thermal cycle requirements for a military EO/IR system.

Validation and testing. A clear strategy should be provided to validate operation of the Phase 1 prototype and test the multi-function imaging capability, include all traditional metrics for imaging systems in each mode of operation (spectral band coverage, responsivity, noise, NETD, noise equivalent power or irradiance, etc.). Specific objectives of interest for Phase 1 proposals are given in Table 3, and additional metrics relevant to the specific application should be provided. Proposers can assume that both the Gen-1 and Gen-2 ROIC designs will be tested to verify functionality. Testing will be done at a variety of temperatures between 77K and room temperature. Any screening of digital tier ROICs to be used in demonstration camera builds will be the responsibility of TA1 and TA2 performers. Sets of test vectors will be provided that may be used for performing basic functional tests of the digital tier.

Proposers should describe each mode of operation in detail. For each mode of operation, proposals should include the following, as well as any other details relevant to the application:

Detector type Spectral response Pixel circuit configuration and features in the analog front end Pixel format (e.g. single pixel, sub-array, or full frame operating mode) Frame rate Data type (intensity, event, time, etc.)

Maximum required bandwidth on the Tier 1 – Tier 2 data I/O interface In pixel processing Perimeter processing Optics configuration (if variable)

Phase 1 Design Phase 1 Design is a nine month activity, with a second nine month option period. All proposals are expected to include Phase 1 Design activity. The objective is to continue the work described in Phase 0, but now focus on developing a detailed operational description and simulation for the Gen-2 digital layer. Proposers should describe specific enhancements to Table 1 that would be required for Gen-2 applications, if there are any.

Phase 1 Design performers can expect iterative distribution of software from the government incorporating updated Gen-2 framework models. The generalized power consumption framework developed during Phase 0 should be developed into a detailed power analysis specific to the Phase 2 design and include a detailed thermal analysis. During the Extended Option period, performers should begin design and layout of the Tier 2 layer, including the fabrication of test chips, if appropriate, for the proposed technology. During the Extended Option period, performers will develop a detailed plan for the Gen-2 prototype that includes optics, detector, electronics, and a detailed test plan.

Phase 2 - Option The objective of Phase 2 is the demonstration of significant improvement over the state of the art in terms of reconfigurability, functionality, and the ability to provide move valuable information from a single FPA. While Gen-1 applications may be restricted by the existing resources in the digital layer, proposals will have the opportunity to recommend specific features and capabilities in the Gen-2 ROIC. New and unique features that are necessary for Phase 2 should be highlighted in the proposal, but should be justified in terms of real estate, power consumption, and the ability to generalize these resources for multiple applications. Proposers should also describe the anticipated limitations of the Gen-1 layer for their proposed application, and quantify the benefits of a more advanced design.

With respect to operating modes, performance objectives, and technical approach, Gen-2 Demonstration proposals should address the same topics as Gen 1 Demonstration proposals, and should highlight advances made between Phases 1 and 2 demonstrations. While Gen 1 demonstrations can be laboratory based, Gen 2 demonstrations should be portable and suitable for testing outdoors. However, extensive custom electronics and packaging of the camera should not be included in Phase 2 proposals.

Government Furnished Property (GFP) and Information: The following items and data can be anticipated to be furnished by the Government to teams selected for Phase 1 demonstrations.

Dates are counted from the beginning of the program. These dates are approximate but should be used for planning purposes:

Phase 0 (Month 1):

o Specification sheet for the Gen-1 Tier 1 IC o Interface control document (ICD) for Tier 1 to Tier 2, and Tier 1 to electronics o CAD flow software and supporting documentation for the Gen-1 IC o Benchmark models for simulation, bitstream generation, and configuration

Phase 1 Demonstration (Month 18) o Tier 1 die, in the form of (2) 300 mm wafers o Development kit. Performers will receive a physical development kit that will provide an electronic interface to the Tier 1 IC. A development board will be included with a cable interconnect to the FPA. Performers will be responsible for the design and fabrication of a daughter board to house the FPA that is appropriate for their test environment. The development kit will enable prototyping and demonstration of the proposed operational models in hardware.

Phase 1 Design (Month 10) o Specification sheet for the Gen-2 Tier 1 IC o ICD for Tier 1 to Tier 2, and Tier 1 to electronics o Tier 1 (Gen 2) simulation software and supporting documentation o Benchmark models for simulation, bitstream generation, and configuration

Phase 2 (Month 33) o Tier 1 die, in the form of (2) 300 mm wafers o Gen 2 development kit, similar to the Gen 1 kit described above

Technical Interchange Meetings: Performers will be expected to spend a minimum of one and a maximum of three days at a Government site for quarterly technical interchange meetings (TIMs) with the Tier 1 hardware and software design team. Kickoff and end of phase program reviews will supplant these TIMs. The objectives of the TIMs will include providing an update to performers on hardware designs and software, and for performers to update the Government team with concepts, simulation results, plans, and application requirements.

Performance Objectives: The ReImagine platform endeavors to enable revolutionary capabilities across a wide range of applications. As such, proposers must provide application-relevant performance metrics. It is required that proposals describe measurable, quantitative milestones at the conclusion of each phase. The templates given below for performance objectives are provided as guidelines, but following them exactly is not required for proposal compliance, due to the variation in expected proposals. All proposals should provide a technical rationale for the approach and program milestones, as well as a clear trajectory to achieving the end of program goals.

Table 3 gives objectives that should be met or defined for the Phase 1 and Phase 2 demonstrations. Note that additional information is requested for TA2 proposals. Specifically, TA2 proposals are of interest that incorporate passive imaging together with one or more of Geiger mode direct detection, high gain linear mode detection with near single photon noise levels, or high bandwidth AC coupled linear mode coherent operations. Proposals that use active functionality for measurements other than distance, e.g. laser vibrometry, are also of interest and should apply to TA2. Modes should allow for high data rate active ROIs, including burst mode operation. The potential for streamed processing and data compression on chip is also of interest.

Based on each mode of operation, proposers should benchmark the state of the art (SOA), and compare the implementation in ReImagine to the most relevant figures of merit (FOM) for that application. An example template is given in Table 4. If a particular FOM will fall short of the SOA, proposers should explain why this will not impact the overall capabilities for that application. The information provided in response to Table 4 is key to a successful proposal, as it embodies the impact that ReImagine will have on capabilities. Of particular interest to ReImagine are proposals that either a) combine SOA capabilities to achieve a mission objective that was previously not possible with a single focal plane, or b) demonstrate new methods for collecting and processing data that leverage the reconfigurability and/or embedded processing of the ReImagine digital layer.

Table 3. TA1/TA2 camera configuration and performance objectives.

Phase 1 Phase 2 Spectral Band(s) (µm) Proposer Defined Proposer Defined Pixel Pitch (µm) ≥ 10 µm ≥ 8 µm Array Format 640 × 512 1280 × 1024 Imaging mode (TA1 and TA2)

Distinct imaging modes of operation ≥ 3 (TA1)

≥ 2 (TA2)

≥ 4 (TA1)

≥ 3 (TA2)

LSB (e-) Proposer Defined Proposer Defined Front end noise (e-) Proposer Defined Proposer Defined Range mode (TA2 only) Front end bandwidth (GHz) Proposer Defined Proposer Defined Range Precision (m) Proposer Defined Proposer Defined Crosstalk (%) Proposer Defined Proposer Defined Photon collection efficiency Proposer Defined Proposer Defined Noise equivalent photons Proposer Defined Proposer Defined Minimum time between events (µs) Proposer Defined Proposer Defined

Table 4. Objectives for TA1 and TA2.

Distinct modes of operation

SOA capability Gen 1 Gen 2

Mode 1 FOM 1:

FOM 2:

Etc.

FOM 1:

FOM 2:

Etc.

FOM 1:

FOM 2:

Etc.

Mode 2 FOM 1:

FOM 2:

Etc.

FOM 1:

FOM 2:

Etc.

FOM 1:

FOM 2:

Etc.

Etc. Etc. Etc. Etc.

Deliverables: All Technical Area 1 and Technical Area 2 performers shall be required to provide the following deliverables:

Technical reports for all kickoff and program review meetings Technical reports from quarterly TIMs Monthly financial reports Phase 0 o Detailed report on reconfigurable applications and Phase 1 plans o Verilog files for Gen-1 configurations

Phase 1 Demonstrations o Gen-1 Tier 2 Preliminary Design Review (PDR) and Critical Design Review (CDR) documents o Gen-1 camera PDR and CDR documents o Verilog files for Gen-1 configurations o One prototype camera, delivered to a government facility, meeting the final Phase 1 objectives Phase 1 Design o Detailed report on reconfigurable applications and plans for Phase 1 prototype o Verilog files for Gen-2 configurations

Phase 1 Design Option o Gen-2 Tier 2 PDR documents o Gen-2 camera PDR documents

Phase 2 o Gen-2 Tier 2 CDR documents o Gen-2 camera CDR documents o Verilog files for Gen-2 configurations o One prototype camera, delivered to a government facility, meeting the final Phase 2 objectives

2. Technical Area 3

Use of either a procurement contract, grant, cooperative agreement or other transaction (OT) award instrument is permitted for TA3.

It will be possible for the ReImagine architecture to change modes through user control, or through preset routines. However it is also possible to envision an autonomous system that configures the sensor to collect the most relevant data based on context in the scene. The goal of TA3 is to develop adaptive learning algorithms that guide the sensor, through the real-time adaptation of sensor control parameters, to collecting the data with the highest content of useful information.

TA1 and TA2 will provide only a small number of instantiations of a reconfigurable image sensor, and many others could be conceived. Therefore TA3 will not use ReImagine hardware or software provided as GFP/Information, or developed in TA1 or TA2. Instead, TA3 efforts are expected to develop camera models in software that can explore design parameters that may guide the development of future reconfigurable sensors. Proposers should assume the availability of only input data that comes from the reconfigurable image sensor, and will be responsible for generating that data during the program. In contrast to sensor fusion, models must consider not only the relative value of data, but also the opportunity cost of data not being collected at a given location and time.

Proposals to TA3 must exhibit two strengths: capability advancement and implementability.

Capability advancement must be defined by proposers in terms of objectively observable, numerical metrics. Proposers should specify their own metrics. Proposers must specify baseline values for these metrics using one or multiple fixed sensors, and values that they propose to achieve at each ReImagine phase boundary. Proposers must specify how these metrics will be measured on their final, and in-development, algorithms. The test data sets that are proposed for use in these measurements shall be specified. Measurable milestones must be proposed by phase.

Implementability measures the degree to which the proposed algorithms can be used in realistic systems. TA3 performers are not limited to the functions that can be anticipated from the TA1-2 Technical Areas. Real-time parametric control over spatial resolution, temporal resolution, spectral response, and polarization response may be considered, as well as the use of distance measurements. Other configurable parameters may be proposed. However the capabilities of the sensor must be bounded and specified. Requirements for the algorithms must be constrained within reasonable bounds; for example, if machine learning algorithms are proposed, then the required amount of labeled and unlabeled training data must be estimated.

TA3 will consist of three 12-month phases (Base and two options), with a start date coincident with Phase 0 of TA1-2, as shown in Figure 2. TA3 performers will not receive Government Furnished Property or Information during the program. It is anticipated that the scope of TA3 efforts will be fundamental research and, as such, publication of the results developed under TA3 will be encouraged. Dissemination of results at program-wide meetings will be required. See the “Fundamental Research” section below for more information.

Phase 1 – Base Period Phases 1 and 2 should focus primarily on the Capability Advancement criteria. In Phase 1, proposers are expected to develop a detailed framework and begin implementation of the proposed algorithms. This may consist of a reduced complexity model in terms of configurable parameters relative to the ultimate capability of the proposed sensor model. Proposers should provide specific quantitative milestones for the algorithms, as well as a clear description of the methodology for their evaluation. At the end of Phase 1, performers must demonstrate streaming operation of the algorithm with a 1000x slowdown in real time.

Phase 2 - Option In Phase 2 proposers should develop a complete model for the reconfigurable sensor and explore variations on the fundamental algorithm. Performers must develop a complete test data set that covers the full parameter space to evaluate the performance of the algorithms against the proposed criteria. At the end of Phase 2 performers must demonstrate streaming operation of the algorithm with a 10x slowdown in real time.

Phase 3 - Option In Phase 3 performers should focus on Implementability. Feedback learning algorithms should inherently provide real-time decisions to the sensor. Considering this proposers should propose objectives that reduce the size, weight, and power of the computation resources required to run their algorithms in real time. Operation should be demonstrated in a real-time camera model that responds to feedback from their algorithms and demonstrates the full Capabilities Advancement demonstrated in Phases 1-2.

Deliverables All Technical Area 3 performers shall be required to provide the following deliverables:

Technical reports for all kickoff and program review meetings Quarterly technical reports Monthly financial reports

II. Award Information

Multiple awards are anticipated. The amount of resources made available under this BAA will depend on the quality of the proposals received and the availability of funds.

The Government reserves the right to select for negotiation all, some, one, or none of the proposals received in response to this solicitation, and to make awards without discussions with proposers. The Government also reserves the right to conduct discussions if it is later determined to be necessary. If warranted, portions of resulting awards may be segregated into pre-priced options. Additionally, DARPA reserves the right to accept proposals in their entirety or to select only portions of proposals for award. In the event that DARPA desires to award only portions of a proposal, negotiations may be opened with that proposer. The Government reserves the right to fund proposals in phases with options for continued work at the end of one or more of the phases.

Awards under this BAA will be made to proposers on the basis of the evaluation criteria listed below (see section labeled “Application Review Information,” Sec. V.), and program balance to provide overall value to the Government. The Government reserves the right to request any additional, necessary documentation once it makes the award instrument determination. Such additional information may include but is not limited to Representations and Certifications. The

Government reserves the right to remove proposers from award consideration should the parties fail to reach agreement on award terms, conditions and cost/price within a reasonable time or the proposer fails to timely provide requested additional information. Proposals identified for negotiation may result in a procurement contract, grant, cooperative agreement, or other transaction, depending upon the nature of the work proposed, the required degree of interaction between parties, whether or not the research is classified as Fundamental Research, and other factors.

In all cases, the Government contracting officer shall have sole discretion to select award instrument type and to negotiate all instrument terms and conditions with selectees. Proposers are advised that regardless of the instrument type proposed, DARPA personnel, in consultation with the Government contracting officer, may select other award instruments, as they deem appropriate. DARPA will apply publication or other restrictions, as necessary, if it determines that the research resulting from the proposed effort will present a high likelihood of disclosing performance characteristics of military systems or manufacturing technologies that are unique and critical to defense. Any award resulting from such a determination will include a requirement for DARPA permission before publishing any information or results on the program. For more information on publication restrictions, see the section below on Fundamental Research.

Fundamental Research

It is DoD policy that the publication of products of fundamental research will remain unrestricted to the maximum extent possible. National Security Decision Directive (NSDD) 189 established the national policy for controlling the flow of scientific, technical, and engineering information produced in federally funded fundamental research at colleges, universities, and laboratories. The Directive defines fundamental research as follows:

'Fundamental research' means basic and applied research in science and engineering, the results of which ordinarily are published and shared broadly within the scientific community, as distinguished from proprietary research and from industrial development, design,…