HR001120S0045 Invisible Headlights_Amendment 3 (tracked changes).pdf

HR001120S0045 INVISIBLE HEADLIGHTS 1

Broad Agency Announcement

Invisible Headlights (IH) Defense Sciences Office

HR001120S0045

Amendment 32

May 2715, 2020

HR001120S0045 INVISIBLE HEADLIGHTS 2

Table of Contents I. Funding Opportunity Description

A. Introduction B. Background C. Program Description/Scope D. Program Structure E. Government-furnished Property/Equipment/Information F. Technical Area Descriptions G. Schedule/Milestones H. DARPA Embedded Entrepreneur Initiative I. Deliverables J. Other Program Objectives and Considerations

II. Award Information A. General Award Information B. Fundamental Research

III. Eligibility Information A. Eligible Applicants B. Organizational Conflicts of Interest C. Cost Sharing/Matching D. Other Eligibility Requirements

IV. Application and Submission Information A. Address to Request Application Package B. Content and Form of Application Submission C. Submission Dates and Times D. Funding Restrictions E. Other Submission Requirements

V. Application Review Information A. Evaluation Criteria B. Review and Selection Process C. Federal Awardee Performance and Integrity Information (FAPIIS)

VI. Award Administration Information A. Selection Notices B. Administrative and National Policy Requirements C. Reporting

VII. Agency Contacts VIII. Other Information

A. Frequently Asked Questions (FAQs) B. Proposers Day

BAA Attachments:

• Attachment A: ABSTRACT SUMMARY SLIDE TEMPLATE

• Attachment B: ABSTRACT TEMPLATE

• Attachment C: PROPOSAL SUMMARY SLIDE TEMPLATE

• Attachment D: PROPOSAL TEMPLATE VOLUME 1 TECHNICAL & MANAGEMENT VOLUME

• Attachment E: MS ExcelTM DARPA COST PROPOSAL SPREADSHEET

• Attachment F: PROPOSAL TEMPLATE VOLUME 2 COST VOLUME_Amendment 1

• Attachment G: PROPOSAL TEMPLATE VOLUME 3 ADMINISTRATIVE & NATIONAL POLICY REQUIREMENTS VOLUME_Amendment 1

HR001120S0045 INVISIBLE HEADLIGHTS 3

PART I: OVERVIEW INFORMATION

• Federal Agency Name: Defense Advanced Research Projects Agency (DARPA), Defense Sciences Office (DSO)

• Funding Opportunity Title: Invisible Headlights (IH)

• Announcement Type: Initial Announcement

• Funding Opportunity Number: HR001120S0045

• Catalog of Federal Domestic Assistance (CFDA) Number(s): 12.910 Research and Technology Development

• Dates (All times listed herein are Eastern Time.)

o Posting Date: April 17, 2020 o Proposers Day: March 16, 2020. See Section VIII.B.

o Abstract Due Date: May 08, 2020, 4:00 p.m.

o FAQ Submission Deadline: June 1003, 2020, 4:00 p.m. See Section VIII.A.

o Full Proposal Due Date: June 2417, 2020, 4:00 p.m.

• Anticipated Individual Awards: DARPA anticipates one or more awards in Technical Area 1 and one or more awards in Technical Area 2.

• Types of Instruments that May be Awarded: Procurement contracts, cooperative agreements or other transactions

• Agency contacts o Technical POC: Joseph Altepeter, Program Manager, DARPA/DSO o BAA Email: InvisibleHeadlights@darpa.mil o BAA Mailing Address:

DARPA/DSO

ATTN: HR001120S0045

675 North Randolph Street Arlington, VA 22203-2114 o DARPA/DSO Opportunities Website: http://www.darpa.mil/work-with-us/opportunities

• Frequently Asked Questions (FAQ): FAQs for this solicitation may be viewed on the DARPA/DSO Opportunities Website. See Section VIII.A for further information.

http://www.darpa.mil/work-with-us/opportunities?tFilter=&oFilter=2&sort=name http://www.darpa.mil/work-with-us/opportunities?tFilter=&oFilter=2&sort=name

HR001120S0045 INVISIBLE HEADLIGHTS 4

HR001120S0045 INVISIBLE HEADLIGHTS 5

PART II: FULL TEXT OF ANNOUNCEMENT

I. Funding Opportunity Description

This Broad Agency Announcement (BAA) constitutes as a public notice of a competitive funding opportunity as described in Federal Acquisition Regulation (FAR) 6.102(d)(2) and

35.016 as well as 2 C.F.R. § 200.203. Any resultant negotiations and/or awards will follow all laws and regulations applicable to the specific award instrument(s) available under this BAA, e.g., FAR 15.4 for procurement contracts.

A. Introduction

The Defense Sciences Office (DSO) at the Defense Advanced Research Projects Agency (DARPA) is soliciting innovative research proposals in the area of passive three-dimensional (3D) vision using ambient thermal emissions. Proposed research should investigate innovative approaches that enable revolutionary advances in science, devices, or systems. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice.

B. Background Autonomous and semi-autonomous systems require active illumination to navigate at night or underground. This creates a significant vulnerability, as this active signature may be detected by adversaries kilometers beyond the useful sensor range. The Invisible Headlights program will develop solutions to eliminate this vulnerability by discovering how to exploit the ambient thermal light present in all environments using a totally passive 3D sensor that is accurate enough and fast enough to support autonomous navigation.

The Invisible Headlights approach is fundamentally different than previous efforts aimed at target recognition using infrared emissions. Conventional infrared sensors fail, by design, to collect almost all available information from ambient thermal emissions. Under the Invisible Headlights approach, non-target artifacts in the scene which might previously have been considered “clutter” will instead provide the signal needed to enable 3D vision. Because the utility of this approach is limited to the information content available in real scenes, the Invisible Headlights program will quantify the available information in ambient thermal emissions, determine how much of that information is useful for building a 3D model of a scene, define the tradespace of sensor designs capable of gathering this information, develop new sensors capable of dramatically increased measurement diversity, and validate all of the above via testing in the field.

In principle, ambient thermal emissions can be used to generate a 3D model of the environment using totally passive sensors. There are, for example, several approaches that might use thermal emissions to passively generate such 3D models. These include stereo ranging, optical flow, and atmospheric-absorption-based ranging. Each of these has advantages and drawbacks. For example, stereo ranging uses two or more cameras and triangulation to estimate the distance to a feature that is visible to both cameras. Optical flow exploits the same basic physics of triangulation using a single moving camera. Both methods have been demonstrated using monochrome thermal cameras, but these demonstrations often suffer from low image contrast;

HR001120S0045 INVISIBLE HEADLIGHTS 6

triangulation cannot work unless both views of a scene contain a common image feature.

Even in scenes with very low contrast, atmospheric-absorption-based ranging can be used to estimate the range to an object by inferring the amount of atmosphere through which emitted thermal light must have traveled. This is analogous to inferring the distance to an object that is barely visible in a fogbank. For a given density of fog, a given amount of obscuration corresponds to a given distance. Given the huge range of absorptivity in the infrared spectrum, for any given distance there may be a particular wavelength that corresponds to just the right “density of fog.”

For each of these approaches, it remains an open question whether the ambient thermal emissions, particularly emissions present in the myriad off-road environments into which autonomous vehicles might one day be deployed, provide sufficient useful information. Real objects do not act as perfect black or gray bodies, nor do they exhibit the high visibility spectral contrast that might be measured in a spectroscopy lab; spatial, spectral, and polarimetric features are instead “washed out” by reflected and re-reflected thermal emissions from the entire scene.

Sensors capable of measuring more polarizations, wavelengths, and spatial modes per unit time – in other words, sensors capable of increased measurement diversity – could provide superior triangulation-based and atmospheric-absorption-based 3D vision. For triangulation, increased measurement diversity has the potential to increase contrast in a scene. Improved spectral resolution allows an imager to always find just the right “density of fog” to enable atmospheric-absorption-based 3D vision. Unfortunately, most conventional mid-wave infrared (MWIR = 3-5 µm) and long-wave infrared (LWIR = 7-15 µm) imagers utilize two-dimensional (2D) focal plane arrays that measure a single broad color band or 2D focal plane arrays where one axis maps to spatial modes and one axis maps to measured wavelength bands. Completely new types of sensors may be needed to enable dramatic increases in measurement diversity.

Note that triangulation-based and atmospheric-absorption-based 3D vision are used as examples only; proposers to the Invisible Headlights program are not limited to these approaches and are encouraged to propose better alternatives.

C. Program Description/Scope The primary goals of the Invisible Headlights program are to

1. understand the useful information contained in ambient thermal emissions,

2. enable passive 3D vision for autonomous navigation.

Understanding ambient thermal emissions. Quantifying the measurable information from ambient thermal emissions is non-trivial and very environment-dependent. It depends on the temperature, chemical composition, geometry, and atmosphere surrounding the objects in a scene. Moreover, it depends on the aperture, position, measurement modes, and performance of the sensor or sensors. In practice, these environments are so complex that measurement, rather than modeling, is needed to characterize the actual signal variation as a function of spatial mode, polarization, and wavelength. However, the only hard limit physics places on this information content is shot noise. In principle, an ideal sensor might be able to extract many orders of

HR001120S0045 INVISIBLE HEADLIGHTS 7

magnitude more data about the environment than is attainable using conventional infrared sensors. Invisible Headlights performers will quantify, characterize, and visualize the information content in a variety of outdoor environments. In addition, the Invisible Headlights program will determine how to exploit ambient thermal emissions for revolutionary technological advances.

Enabling passive 3D vision for autonomous navigation. The specific application of 3D vision for autonomous navigation can be used to derive clear performance metrics, such as spatial resolution, vision distance, and measurement time, which scale aggressively with vehicle speed (e.g., stopping distance scales quadratically as a function of speed). Therefore, the Invisible Headlights program will use passive 3D vision as a single common application to focus performers’ research efforts. Because 3D vision from thermal sensors will have very different performance scaling and limits than light detection and ranging (LIDAR), it is possible that this approach could lead to a transformative new sensing technology for autonomous systems.

Particularly for high-speeds, thermal sensors that enable 3D vision for autonomous navigation are expected to require near-zero noise and orders of magnitude greater measurement diversity than conventional sensors. This will likely require the development of completely new types of infrared sensors. Although the program metrics are focused on 3D vision for ground-based autonomy, sensors focused on other autonomy modes may be modeled and explored as part of the program.

The Invisible Headlights Program will be executed in three phases and development efforts will be divided into four primary areas:

• Technical Area 1: Near-Term Passive 3D Vision o In recent years, dramatic advances have been made in infrared focal plane technology; performer(s) in this Technical Area will modify and extend existing sensors in order to enable passive 3D vision sensors that have the potential for low-speed applications (speeds up to 25 mph).

• Technical Area 2: Novel Sensors for Measurement Hyperdiversity o Performer(s) in Technical Area 2 will develop near-zero-noise sensors capable of orders of magnitude more measurements per second than conventional sensors, enabling fast, high-spatial-resolution, high-spectral-resolution measurement of an environment, and high-speed 3D vision. Because this will be an entirely new class of infrared sensors, Technical Area 2 is expected to be a fundamental research activity. If successful, these new sensors will enable 3D vision for high-speed autonomy (speeds greater than 25 mph).

• All Performers: Virtual Tradespace Exploration o The single most important consequence of the Invisible Headlights program will be a quantitative understanding of the information contained in ambient thermal emissions, coupled with quantitative insight into what types of sensors can effectively exploit that information. Performers in TAs 1 and 2 will provide this understanding and insight by creating a quantitative tradespace of 3D vision performance as a function of widely varying sensor specifications.

• Government Test and Evaluation (T&E) Team: Ground Truth Data Collection (not being solicited under this BAA)

HR001120S0045 INVISIBLE HEADLIGHTS 8

o In addition to providing Test & Evaluation of the sensor capabilities developed under TAs 1 and 2, the Government Team will create an unprecedented database of high spatial resolution and high spectral resolution measurements of ambient infrared emissions in a variety of outdoor environments, referenced to accurate 3D maps measured using LIDAR. This database will directly support the Virtual Tradespace Exploration efforts described above.

D. Program Structure The Invisible Headlights program is a 57-month research and development effort comprising three phases, as illustrated in the figure below. Both Technical Areas will have the same periods of performance.

Figure 1 – High-Level Program Schedule

DARPA will use a phased acquisition approach for the Invisible Headlights program. Proposers should address all three phases and provide details for the Phase 1 (Base) and Phase 2 (Option) efforts as outlined in Section I.F below. Proposers should also provide a Rough Order of Magnitude (ROM) for Phase 3 including ROM costs, a draft statement of work (SOW), and any additional information on anticipated program plans. Proposals that do not address these Phase 3 goals may be deemed nonconforming.

Phase 1 (Discovery) is an 18-month effort to determine if thermal emissions contain sufficient information – in principle – to enable autonomous driving at night or underground. Phase 2 (Optimization) is a 21-month effort that builds on work in Phase 1 by refining models, creating experimental designs, and testing implementations to show that real systems can measure the information needed to enable 3D vision. If the Government elects to continue the program into Phase 2, it may exercise the option on the contract(s) of the selected performer(s) based on Phase 1 performance and the candidate technologies’ potential to meet the Phase 2 program goals and metrics. The Government retains the right to award all, some, one, none, or portions of the proposed Phase 2 options to support promising further technology developments.

Phase 3 of the program (Advanced Prototypes) will be an 18-month effort to build and test passive demonstration systems that can compete with active sensors. Prior to the completion of Phase 2, DARPA intends to issue Phase 3 proposal instructions requesting updated technical and cost proposals for Phase 3. Participation in the competition for Phase 3 is optional and will be limited to Phase 2 performer(s).

HR001120S0045 INVISIBLE HEADLIGHTS 9

Evaluation of Phase 3 proposals will be based on criteria to be specified in the Phase 3 proposal instructions. The Phase 3 evaluation criteria will be consistent with the evaluation criteria in this solicitation, but may be tailored based on the program plan for Phase 3. Phase 3 proposal evaluations will be conducted through a scientific and technical review process in accordance with Section V.B. The Government reserves the right to issue a new solicitation for Phase 3 with a new award instrument if programmatic circumstances dictate.

Participation in Phase 2 does not guarantee funding in Phase 3; progression to the next phase will be contingent on evaluation of Phase 3 proposals and availability of funds.

E. Government-furnished Property/Equipment/Information To support research and development, DARPA will supply performer(s) with a ground truth database of ambient infrared signals. The Government T&E Team will simultaneously collect high spatial/spectral resolution infrared imagery and accurate 3D maps using LIDAR. For example, a data collection might consist of stereo MWIR hyperspectral images collected at 1 meter intervals on a dirt road in rolling terrain. Ground truth data will be collected at multiple points throughout the program in multiple representative environments, such as desert, arctic, underground, temperate, and tropical.

Proposers are encouraged to indicate in their proposals what kinds of ground truth data would best enable their research and at which stage in the program they would need such data.

Proposals should be very specific, indicating desired

• spatial resolution, over what field of view;

• spectral resolution, over what wavelength bands;

• depth resolution, over what ranges;

• camera configurations;

• movement resolution or the distance the vehicle-mounted measurement systems are moved between measurements and/or;

• ancillary data such as visible or near-infrared imagery of the scene (be specific).

Proposals should clearly delineate data requirements versus data requests. After the start of the program, the Government T&E Team will work with performer(s) to optimize the Government Furnished Information (GFI) collected to maximize the chances of program success. DARPA reserves the right to determine which data requirements and requests will be collected and supplied to performer teams as GFI.

The Government T&E Team will be responsible for disseminating the ground truth data. The database is expected to be petabytes in size. Therefore, proposers should account in their proposals for the necessary costs to accept, store, and use this volume of data.

The Government T&E Team may choose to prepare two separate ground truth databases. In this case, the primary database will be export-controlled and will be disseminated to performers eligible to receive export-controlled data (see Section III.D for eligibility requirements and proposal instructions related to eligibility). A secondary database will be derived from the

HR001120S0045 INVISIBLE HEADLIGHTS 10

primary database and will not be export-controlled. DARPA is not responsible for any resulting programmatic disadvantage, real or perceived, resulting from ineligibility to access export-controlled data.

F. Technical Area Descriptions The Invisible Headlights program is divided into two Technical Areas (TAs).

1. Technical Area 1 (TA 1) – Near-Term Passive 3D Vision TA 1 will focus on modifying and extending existing sensors and developing related processing and perception techniques to enable passive 3D vision for autonomous navigation. At program kickoff, state-of-the-art sensors will be “3rd Generation FLIR.” These sensors have large focal plane arrays, fast update rates, shot-noise-limited performance, and either single-band or few-band frequency sensitivity in the mid-wave-infrared (MWIR) or long-wave-infrared (LWIR).

Proposers to TA 1 are encouraged to explore the utility of these state-of-the-art sensors as well as other lower-cost, lower-performance sensors. Sensor variety will enable the program to determine exactly how 3D vision performance scales with sensor quality in this regime.

Proposers are encouraged to propose novel methods for achieving passive 3D vision.

Performer(s) in TA 1 are not required to tie their approach to increased measurement diversity (i.e., measuring more colors, spatial modes, and polarizations); any effective technical approach that can meet the program metrics (see metrics in Section I.G) will be considered. Success in TA 1 would enable a near-term, totally passive 3D vision system for off-road autonomous ground vehicles traveling at speeds nominally up to 25 mph.

Because performer approaches are expected to be technically diverse, proposers are permitted to propose their own schedule of technical and programmatic milestones that support achieving the required program metrics. All proposals to TA 1 are, however, required to address the following essential questions:

• Which existing sensors will be modified as part of this program? What are their relevant specifications? Be specific and complete.

• How will these sensors be modified? What are the time and schedule implications of these modifications?

• What key design decisions do you expect will be necessary at what point in the program?

• What assumptions are you making about the thermal environment? Which assumptions are required for success?

• What are the limits of your approach?

Sensors that require cooling to as low as 77 K are allowed as part of TA 1, although proposers that wish to explore this type of cooled detector are encouraged to also explore less expensive, lower size, weight, and power (SWaP), higher temperature systems, in order to be able to validate the virtual tradespace in a different regime.

Specifically discouraged are proposals that focus effort on real-time data processing at the expense of accomplishing the primary program goals. Real-time data processing is not required

HR001120S0045 INVISIBLE HEADLIGHTS 11

as part of this program, although real-time measurement is required for some of the program milestones. In other words, for some program milestones, data must be collected quickly (in real time), but the algorithms that transform that data into 3D maps of the environment can be executed offline and at a much slower speed. Development of vehicle autonomy algorithms is not a part of this program; efforts should instead focus on developing passive 3D vision with metrics inspired from ground-based autonomy.

2. Technical Area 2 (TA 2) – Novel Sensors for Measurement Hyperdiversity TA 2 will focus on creating completely new types of sensors capable of measurement hyperdiversity, which is defined here as “capable of performing a number of measurements per second that exceeds the state of the art by multiple orders of magnitude.”

A single measurement is defined here as the incident photons/time (or equivalently, optical power) that fall within a defined spatial or frequency bandwidth and polarization. A single measurement could be a single quantum mode or the sum of many such modes. For example, a polarization-insensitive, Megapixel-size, broadband-LWIR focal plane array operating at a frame rate of 500 Hz would perform 5x108 measurements/second. If the same device was modified such that every pixel registered 1000 different color bands and two different polarizations, it would perform 1012 measurements/second. This example is included for illustrative purposes only and not as a recommended approach. Each proposal should outline a specific approach and combination of measurement techniques that are expected to achieve measurement hyperdiversity.

3D vision that enables high-speed autonomous navigation is expected to require measurement hyperdiversity for the following reasons:

• Minimum stopping distance, and, therefore, required vision distance scale quadratically with speed.

• Higher vision distances imply much higher angular resolution over the same field of view.

• Some 3D vision techniques (e.g., triangulation-based ranging) require increased contrast for feature detection; spectral and polarization diversity are both ways to increase contrast.

• Some 3D vision techniques (e.g., atmospheric-absorption-based ranging) require very high levels of spectral resolution.

• All 3D vision techniques are signal-to-noise ratio (SNR) limited, and increased measurement diversity has the potential to increase SNR in scenes containing high levels of non-redundant information.

TA 2 proposers are not required to propose any particular photon detection technology, but the following detector characteristics are expected to be required to achieve useful signals from measurement hyperdiverse detectors:

• Near-zero levels of noise (e.g., dark counts, read noise, amplifier noise, etc.)

• Very high quantum efficiency, possibly near 100%

HR001120S0045 INVISIBLE HEADLIGHTS 12

• Sensitivity in MWIR and/or LWIR

• A plausible path towards very large detector arrays and very high measurement rates

Initial modeling shows that in order to achieve ground vehicle speeds as high as 75 mph, sensors capable of making ~1012 measurements per second will be needed. All proposers to TA 2 are required to show how their proposed technology can reach this level of measurement hyperdiversity by the end of the program.

Because of the extremely high data bandwidth for a sensor capable of making 1012 measurements per second, TA 2 proposers must describe a realistic plan to create a system that can reduce input data in real time to the essential information that enables 3D vision. For example, a particular sensor might measure many wavelength bands but dynamically select a subset of bands that are actually processed during one particular unit of time. Solutions creating a massive amount of data that cannot realistically be used are not of interest.

Fundamental research activities are appropriate for proposal to TA 2, including technologies that require, for example, high SWaP and/or cryogenic cooling. As long as proposers can successfully argue that a candidate technology can meet the program metrics, no specific levels of performance are required to have been achieved at the time of proposal submission. Although TA 2 metrics are derived from ground-based autonomous systems, success could impact a variety of future applications in autonomy and sensing. Proposers are encouraged to point out where their proposed technology could have a broader transformative impact.

Because performer approaches are expected to be technically diverse, proposers are permitted to propose their own schedule of technical and programmatic milestones that support achieving the required program metrics. All proposals to TA 2 are required to address the following questions:

• What is the state of the art for the type of sensor you wish to explore?

• What is the path you expect to take to advance this detection system from the current state of the art to the required program milestones?

• What is the anticipated noise performance of your proposed system, and can this level of noise support the required level of measurement hyperdiversity required at the end of Phase 3, while still achieving a useful signal to noise ratio?

• What is your technical strategy for measurement hyperdiversity?

• Is your detector a photon counting system?

• Describe the path, if any, to take your system to a low-SWaP, fieldable device after the completion of the program?

• What is the bandwidth over which your detector technology is sensitive?

• What key design decisions do you expect will be necessary at what point in the program?

• What assumptions are you making about the thermal environment? Which assumptions are required for success?

• What are the limits of your approach?

Sensors that require cooling to as low as 1 K are allowed as part of TA 2. No explicit SWaP requirements are included for TA 2. Sensor technologies that are at a research and development

HR001120S0045 INVISIBLE HEADLIGHTS 13

stage are explicitly allowed as part of TA 2, as long as they have the potential to be transformative if successful and there is a path to achieving the required program milestones.

Real-time data processing is not required as part of this program, although real-time measurement is required for some of the program milestones. Development of vehicle autonomy algorithms is not a part of this program; efforts instead focus on developing passive 3D vision with metrics inspired from ground-based autonomy.

3. Virtual Tradespace Exploration Performers in both TAs 1 and 2 will be required to explore and characterize the tradespace of achievable 3D vision performance as a function of infrared sensor specifications. Using the ground truth database filtered through a virtual sensor, performers will demonstrate the efficacy of widely varying approaches by characterizing 3D vision performance as a function of the virtual sensor space.

The tradespace of 3D vision performance as a function of sensor specifications is a huge, multi-dimensional space that relies on a hyper-spectral-resolution, high-spatial-resolution, multi-polarization scene. Performers will be expected to develop advanced techniques for quantifying, visualizing, and understanding the thermal environment along a given route. Similarly, they will be expected to develop advanced techniques for quantifying, visualizing, and understanding the tradespace of 3D vision performance as a function of environment and sensor specifications.

Performers will be required to optimize the tradespace in order to minimize the number of required measurements to achieve a given performance, as well as to minimize the data bandwidth from the detection system to a virtual data processing system.

The goal of the virtual tradespace study is distinct from the effort in each TA; it should help reveal the spectrum of possible approaches, rather than examining one approach in detail. A specific technology proposed under TA 1 or TA 2 may fit inside this tradespace, but the tradespace itself is expected to be much larger. The study must help illuminate the fundamental limits that physics imposes on the problem and the space of possible approaches.

All performers are required to produce increasingly robust and useful virtual tradespace algorithms as the program progresses. Three broad classes of virtual tradespaces are described below, characterized by the types of independent and dependent variables they support.

Performers are required to develop examples of each class of tradespace on the schedule listed in Table 1 below. Proposers should assemble a team with the necessary expertise to execute towards these goals, and provide a clear plan to address the virtual tradespace program goals.

Point-Design Virtual Tradespace. This class of virtual tradespace algorithms predicts the vehicle speed enabled by a specific sensor design and specific sensor configuration used in a specific type of environment. Because the experimental systems investigated in each TA are so different, it is anticipated they will provide valuable complementary experimental tests of the Point-Design Virtual Tradespace algorithms.

• Independent Variables (Inputs) o Sensor Specifications: To include measurement types, measurement rates, sensor noise profile, etc.

HR001120S0045 INVISIBLE HEADLIGHTS 14

o Sensor Configuration: Sensor number and geometry. For example, two sensors in a stereo configuration 1.5 meters apart.

o Ambient Infrared Environment: As characterized by, for example, mean temperature, temperature variance, spectral contrast, etc.

• Dependent Variables (Outputs) o Achievable Vehicle Speed: What is the enabled autonomous vehicle speed?

Assume a minimum stopping distance given by the “vision range” listed in Table 3.

o 3D Vision Approach: What approach supports the reported vehicle speed?

Examples include atmospheric-absorption-based reconstruction or stereo-based reconstruction.

o Detailed Performance Metrics: What detailed 3D vision performance metrics support the reported vehicle speed? See Table 3 for an example of detailed metrics supporting a LIDAR-like reconstruction technique.

Multiple-Design Virtual Tradespace. This class of virtual tradespace algorithms predicts the optimal sensor design that maximizes achievable vehicle speed given certain design constraints.

• Independent Variables (Inputs) o Maximum Number of Measurements per Second: As defined in Section F.2.

Assume the sensor must always perform the same measurements.

o Sensor Noise Profile: Sensor noise associated with each measurement.

o Maximum Data Bandwidth: Assume raw measurements must be compressed before data is transmitted to the central processing unit (CPU), and the data link connecting the detector to the CPU is limited by this value. This limitation approximates the design constraints imposed by real read-out integrated circuits (ROICs).

o Sensor Configuration: Sensor number and geometry. For example, two sensors in a stereo configuration 1.5 meters apart.

o Ambient Infrared Environment: As characterized by, for example, mean temperature, temperature variance, spectral contrast, etc.

• Dependent Variables (Outputs) o Sensor Measurements: Which measurements, specifically, optimize achievable vehicle speed within the given constraints?

o Achievable Vehicle Speed: What is the enabled autonomous vehicle speed?

Assume a minimum stopping distance given by the “vision range” listed in Table 3.

o 3D Vision Approach: What approach supports the reported vehicle speed?

Examples include atmospheric-absorption-based reconstruction or stereo-based reconstruction.

o Detailed Performance Metrics: What detailed 3D vision performance metrics

Adaptive-Design Virtual Tradespace. This class of virtual tradespace algorithms predicts the optimal adaptive sensor strategy that maximizes achievable vehicle speed given certain design constraints.

HR001120S0045 INVISIBLE HEADLIGHTS 15

• Independent Variables (Inputs) o Maximum Number of Adaptive Measurements per Second: Assume the sensor can dynamically adjust which measurements are performed from moment to moment, e.g., a sensor that always measures three color bands but is capable of dynamically adjusting the central frequencies of the three color bands.

o Sensor Noise Profile: Sensor noise associated with each measurement.

o Maximum Data Bandwidth: Assume raw measurements must be compressed before data is transmitted to the central processing unit (CPU), and the data link connecting the detector to the CPU is limited by this value. This limitation approximates the design constraints imposed by real ROICs.

o Sensor Configuration: Sensor number and geometry. For example, two sensors in a stereo configuration 1.5 meters apart.

o Ambient Infrared Environment: As characterized by, for example, mean temperature, temperature variance, spectral contrast, etc.

• Dependent Variables (Outputs) o Sensor Measurements: Which adaptive measurement strategy, specifically, optimizes achievable vehicle speed within the given constraints?

o Achievable Vehicle Speed: What is the enabled autonomous vehicle speed?

Assume a minimum stopping distance given by the “vision range” listed in Table 3.

o 3D Vision Approach: What approach supports the reported vehicle speed?

Examples include atmospheric-absorption-based reconstruction or stereo-based reconstruction.

o Detailed Performance Metrics: What detailed 3D vision performance metrics

All proposals are required to address the following questions about the virtual tradespace exploration effort:

• What do you anticipate are the best metrics for characterizing the ambient infrared environment, specifically the metrics that should be used as the independent variables in a virtual tradespace? How do you expect to be able to quantify and visualize the information content in the ambient infrared environment?

• What 3D vision approaches do you anticipate informing your virtual tradespace exploration? Stereo? Optical flow? Atmospheric-absorption-based? Other?

• What are the specific 3D reconstruction metrics that you will use to support a specific vehicle speed in the tradespace? See Table 3 for a LIDAR-based example.

• What alternate approaches to 3D reconstruction will you pursue? For example, do you anticipate being able to leverage 3D semantic scene segmentation to understand a scene without first creating a LIDAR-like point cloud?

• How do you believe the approaches you are considering will be fundamentally limited by temperature variation in the scene? Hyperspectral contrast in the scene? Geometry of the scene?

• Are there other fundamental limits you believe impact this problem?

HR001120S0045 INVISIBLE HEADLIGHTS 16

Collaborative efforts/teaming are strongly encouraged, especially considering the multidisciplinary nature of the expected research. Success in this area will result in new autonomous algorithms for deriving desired sensor specifications for specific tasks; for example, a Multiple-Design Virtual Tradespace algorithm could derive the ideal three frequency bands for enabling 30-mph 3D vision in arctic environments, given a particular focal plane array size and sensor noise profile.

4. Proposing to Multiple Technical Areas Proposers wishing to propose to multiple Technical Areas must submit separate technical and cost proposals for each. Proposers should not submit proposals that combine multiple Technical Areas into a single effort; DARPA may deem proposals that address multiple technical areas non-conforming and remove them from consideration. Proposers should strive to provide a clear understanding of the cost, risk, and organizational expertise to be used within each proposed effort.

G. Schedule/Milestones Performers will be evaluated using a number of milestones and metrics as enumerated below.

Attainment of the milestones and metrics for a given phase does not guarantee transition into the next phase of the program. Individual efforts will also be judged on their expected ability to have a transformative impact on DoD and DARPA priorities.

Virtual Tradespace Technical and Management Milestones Proposers must define their own schedule of technical milestones needed to succeed at virtual tradespace exploration. At a minimum, performer(s) are expected to achieve the following milestones by the quarter-after-award indicated:

Table 1 – Virtual Tradespace Exploration Milestones (TA 1 and TA 2)

Quarter After

Award Milestone

Phase 1 1 N/A 2 Quantify/visualize information content in thermal scenes 3 Initial virtual sensor implementation 4 Demonstrate first Point-Design Virtual Tradespace 5 Prepare detailed priority list for Phase 2 ground truth measurements based on tradespace results 6 Demonstrate first Multiple-Design Virtual Tradespace

Phase 2 7 N/A 8 Demonstrate first Adaptive-Design Virtual Tradespace 9 N/A

HR001120S0045 INVISIBLE HEADLIGHTS 17

10 Generate tradespaces tailored to Government-team custom goals (e.g., different modes of autonomy)

11 N/A 12 Deliver improved Point-Design, Multiple-Design, and Adaptive-Design Virtual

Tradespaces informed by both improved ground truth databases and more advanced performer algorithms

13 Work with Government stakeholders and transition partners to maximize impact of developed technologies; continue to optimize algorithms

Phase 3 14-19 Required milestones for Phase 3 will be detailed in Phase 3 proposal instructions issued near the end of Phase 2. For estimating ROM costs, please use the milestones illustrated in Figure 2.

TA 1 and TA 2 Technical and Management Milestones Performer(s) are required to characterize the information content of ambient infrared emissions and to exploit that information for 3D vision. Achievable 3D vision performance is correctly characterized by a number of detailed performance metrics that depend on the specific approach a performer uses to achieve 3D vision. For a LIDAR-like approach, these metrics would include 3D vision distance, transverse resolution, longitudinal resolution (i.e., ranging accuracy), update rate, and points per update. For an approach based on, for example, developing a semantic understanding of the scene, the detailed metrics could be quite different.

Proposers are required to clearly show how their preferred detailed performance metrics support the speeds listed in Table 2 The achieveable vehicle speed metric will be used to characterize the experimental systems created in TAs 1 and 2. Table 2 shows the required metrics for each key program element, using these definitions:

Mobile 3D Vision Sensing that is performed from a continuously moving vehicle.

Stationary 3D Vision Sensing that is performed from a vehicle that occupies a series of stopped positions. The vehicle can stop at a location, sense for as long as required, then move to a new position. Note that in this way, even 3D reconstruction techniques that leverage motion, such as optical flow, can be simulated using “stationary” measurements.

Table 2 – Technical Milestones for TA 1 and TA 2

Milestone Phase 1 Phase 2 Phase 3

TA 1: Near-Term Passive 3D Vision

Stationary 3D Vision @ distance / resolutions supporting listed speed 25 mph

Stationary 3D Vision @ distance / resolutions / time supporting listed speed

25 mph

Mobile 3D Vision @ listed speed 10 mph 25 mph

HR001120S0045 INVISIBLE HEADLIGHTS 18

Demonstrate that operationally relevant SWaP is feasible

YES

TA 2: Novel Sensors for Measurement Hyperdiversity

Stationary 3D Vision @ distance / resolutions supporting listed speed

35 mph

Stationary 3D Vision @ distance / resolutions / time supporting listed speed

50 mph

Mobile 3D Vision @ listed speed 10 mph

Demonstrate that it is feasible to scale sensor to listed measurement diversity / 3D vision speed

50 mph;

1012 meas. / s

Demonstrate listed measurement diversity in real sensor

109 meas. / s 1012 meas. / s

The single-parameter “speed” metric used in Table 2 above is used to summarize a much more complex tradespace. Faster autonomous vehicle speeds impose much more stringent requirements on a supporting 3D vision system, including longer vision distances and more 3D points per update.

Table 3 below shows an example set of performance metrics needed to support various vehicle speeds, assuming a LIDAR-like approach. Proposers are required to provide a similar chart listing detailed performance metrics for their system that support all of the vehicle speeds listed below. Proposers may assume a 3D vision approach is sufficient if it supports a minimum reactive autonomy stopping distance equal to the “vision range” metric in Table 3. The single metric “stopping distance” is used for simplicity, instead of a much more complex set of potential autonomy requirements (e.g., planning distance, turning distance, etc.). This is because Invisible Headlights is focused on passive 3D vision, not on better autonomy algorithms.

Note the physical sensors developed in TA 1 and TA 2 are not required to support all the speeds listed in Table 3, but proposers should still provide detailed performance metrics for all of these speeds in order to clearly communicate the implications of their approach to 3D vision.

Table 3 – Example Relationship between Speed and 3D Vision Metrics for a LIDAR-like 3D Reconstruction Approach

Detailed Metric 10 mph 25 mph 35 mph 50 mph 75 mph 100 mph

Vision Range 5 m 20 m 40 m 80 m 180 m 320 m

Longitudinal Resolution / Range 1%

Transverse Resolution 0.1 m

HR001120S0045 INVISIBLE HEADLIGHTS 19

Measurement Time / Update

100 ms (Note that while proposers may assume that a 3D model of the world does not need to be updated more frequently than every 100 ms, at high speeds individual measurements will likely need to be taken much faster in order to mitigate the negative effects of motion blur)

Points / Update 2.5k 100k 400k 1,600k 8,000k 25,000k

Demonstrations and Experiments The Invisible Headlights program will assess success throughout the program using a series of demonstrations at performer-chosen sites and/or DARPA-specified sites. The goal of the demonstrations is to assess system performance and the candidate technologies’ potential to meet the program milestones and metrics.

• Proposals should describe proposers’ plans for testing their systems at their own laboratory and/or other “proving grounds” of their choice. If a proposer desires to conduct testing or demonstration at a commercial or Government facility, the proposer must include those costs as part of their proposal in accordance with the proposal instructions.

• Demonstrations at performer-chosen sites are expected to be one day in length.

• Demonstrations at DARPA-specified sites will exercise the system at a field location.

It is likely that the sites to be used will be the same locations collected in the ground truth database, which will allow comparison of real-world and virtual system performance. Proposals should include costs for travel for one day of demonstration and as many days beforehand that the team requires to prepare. For planning purposes, proposer(s) may assume that one experiment will occur at each of the following locations: Twentynine Palms, CA and Fort A.P. Hill, VA.

Other proposal properties

• Proposers should provide a technical and programmatic strategy that conforms to the entire program schedule and presents an aggressive plan to fully address all program goals, metrics, milestones and deliverables.

• The task structure must be consistent across the proposed Schedule, Statement of Work, and Cost Volume.

• A target start date of October 1, 2020 may be assumed for planning purposes.

• Schedules will be synchronized across performer(s), as required, and monitored/revised as necessary throughout the program.

• All proposals must include the following meetings and travel in the proposed schedule and costs:

o To continue integration and development between TAs, foster collaboration between teams, and disseminate program developments, multiple two-day Principal Investigator (PI) meetings will be held in each phase, with locations split between the East and West Coasts of the United States. For budgeting purposes, plan for four (4) two-day meetings over the course of Phase 1 and Phase 2: two (2) meetings in the Washington, D.C. area and two (2) meetings in the San Francisco, CA area. The kickoff meetings should be budgeted for

HR001120S0045 INVISIBLE HEADLIGHTS 20

the Washington, D.C. area.

o Regular teleconference meetings will be scheduled with the Government

Team for progress reporting as well as problem identification and mitigation.

Proposers should also anticipate at least one site visit per phase by the DARPA Program Manager during which they will have the opportunity to demonstrate progress towards agreed-upon milestones.

Figure 2 describes the planned types, locations, and sequencing of meetings and demonstrations:

Figure 2 – Program Event Schedule

H. DARPA Embedded Entrepreneur Initiative (optional sub-section; does not count toward page count) To catalyze the conversion of scientific discovery to impact, the Defense Sciences Office offers applicants the opportunity for additional funding and transition assistance through participation in the Embedded Entrepreneur Initiative. The DARPA Embedded Entrepreneur Initiative (EEI) will provide additional funding, up to $250,000, to employ one entrepreneur-in residence or one corporate business development lead. The entrepreneurial lead's ultimate goal is to develop a robust go to market strategy for entering into defense and commercial markets. The determination for participation in EEI will be made independently by the Government following selection for an award. Selection for award does not imply selection for participation in EEI.

All commercialization and transition activities will be timed to suit the Performer's stage of maturity. Often, the Embedded Entrepreneurial work is most useful in year two or three of a Program. Activities conducted can include, but are not limited to; cost modeling, end user engagement, market analysis and mapping, competitive analysis, techno-economic analysis, manufacturing and scale-up strategy, IP securement strategy, and financial plan creation.

HR001120S0045 INVISIBLE HEADLIGHTS 21

Embedded Entrepreneur participants will work closely with DARPA's Commercial Strategy team and their extensive network of U.S. investors, strategic partners, and mentors.

Proposers wishing to be considered for participation in the Embedded Entrepreneur Initiative must:

• Include an initial hypothesis describing how the proposed technology will transition from its current state to future integration into a product or capability.

• Include a separately costed option in the provided cost proposal template not to exceed

$250,000 that includes tasks describing plans to build and refine a viable Go to Market Strategy over the course of the DARPA program. Tasks contributing to the build of a robust Go to Market Strategy can include, but are not limited to; cost modeling, end user engagement, market analysis and mapping, competitive analysis, techno-economic analysis, manufacturing and scale-up strategy, IP securement strategy, and financial plan creation. These EEI tasks and deliverables must be provided as an option in the Statement of Work submitted in response to this BAA.

• The DARPA EEI option will be exercised at the discretion of the Government based on a performer’s technical accomplishments and progress towards development of a technology with transition and commercialization potential and subject to the availability of funding.

Participation in the Embedded Entrepreneur Initiative is voluntary but highly recommended.

Participants are not expected to form a new company or leave their current research positions to pursue transition, but are expected to, throughout the lifecycle of the proposed effort, identify appropriate partners for enabling transition. Embedded Entrepreneur Initiative funding requests should be consistent with the proposed work scope and proposed timeline, but are anticipated to be in the range of $250,000 per Performer.

I. Deliverables Performer(s) will be expected to provide at a minimum the following deliverables:

• Comprehensive quarterly technical reports due within ten days of the end of the given quarter, describing progress made on the specific milestones as laid out in the SOW.

• A phase completion…