IARPA-RFI-23-01_FINAL C.pdf

Request for Information – Advanced image simulation and 3D object extraction RFI Number: IARPA-RFI-23-02

Agency: Office of the Director of National Intelligence Office: Intelligence Advanced Research Projects Activity – Office of Analysis

IARPA-RFI-23-02

Request for Information (RFI): Advanced image simulation and 3D object extraction

IARPA is seeking information on established datasets and capabilities to create simulations of remotely sensed imagery, automated extraction of objects, and the construction of associated

3D models from remotely sensed imagery. This RFI is issued for planning purposes only, and it does not constitute a formal solicitation for proposals or suggest the procurement any material, software, tools, data sets, etc. The following sections of this announcement contain details on the specific technology areas of interest, along with instructions for the submission of responses.

Background and Scope:

The application of machine learning to remotely sensed imagery requires training data that accurately represents scenarios of interest. A limitation into the efficacy of automated algorithms for image analysis is the volume of available imagery that directly mimics relevant scenarios and contains objects of interest (F. Kong, 2020). Standard simulation processes for remotely sensed imagery are computationally expensive and time consuming; they are limited in producing the volume and range of images required to produce robust automated analytical algorithms (M. F. Reyes, 2022).

An enhanced simulation capability for remotely sensed imagery to foster automated image interpretation and computer vision algorithm design requires the following attributes:

Ability to inject realistic 3D structures of resolvable features to include: buildings, vehicles, foliage, and natural terrain variation

Ability to simulate a variety of wavelengths of interest (multispectral data)

Ability to extract objects of interest and inject these into simulation tools

Rapid simulation capability to produce a large volume of imagery at reasonable cost and time frame

Accuracy from the perspective of human and machine visual systems

The purpose of this RFI is to identify existing image simulation capabilities and ongoing research efforts to create synthetic imagery from satellite and aerial platforms that meet these requirements.

Responses to this RFI should answer any or all of the following questions:

1. Does your organization currently produce high-fidelity image simulation from satellite or high altitude aerial platforms commercially? Is your organization executing research to enable such a capability?

2. What true imagery is utilized to inform the simulation and validate that it is a faithful emulation? What metrics are used to evaluate the overall simulation accuracy and fidelity?

3. What is the range of wavelengths, altitudes, and camera geometries over which the imagery can be simulated? What tunable sensor parameters are included in the simulation model?

4. What is the range of times of day that can be simulated?

5. Beyond Panchromatic Electro-Optical imagery, what other types of remotely sensed imagery can be simulated? What is approximate ground footprint of the simulated imagery? What is the wall clock time required to generate an image? What are the hardware requirements for the simulation capability?

6. What training efforts for machine learning algorithms have been conducted using these simulations? How have the algorithmic results on simulated data been validated?

7. From a true image, can an object of interest be extracted and used to predict an object wire-frame, point cloud,wire frame or other model?

8. Can the simulation capability accept the injection of wire frame models and predict the image response to the terrain model and injected three dimension object model?

9. How is error propagation modelled and visualized through the image simulation?

Preparation Instructions to Respondents:

IARPA requests that submittals briefly and clearly describe the approach or capability, directly address any or all of the specific questions, and outline any known critical technical issues/obstacles. If appropriate, respondents may also choose to provide a non-proprietary rough order of magnitude (ROM) estimate regarding what such approaches might require in terms of funding and other resources for one or more years to support IARPA image simulation needs. This announcement contains all of the information required to submit a response. No additional forms, kits, or other materials are needed.

IARPA welcomes responses from all capable and qualified sources from within and outside of the U.S.

Because IARPA is interested in an integrated approach, responses from teams with complementary areas of expertise are encouraged.

Submissions from Federally Funded Research and Development Centers (FFRDCs) and

University Affiliated Research Centers (UARCs) are permitted but with an understanding that neither groups are able to propose against any future IARPA program. Instead, any submissions from these groups should consider the technical elements described above but include reflection upon how they would support program efforts as a potential test and evaluation partner enabling IARPA to validate different potential approaches to meet the research challenge.

Responses have the following formatting requirements:

1. A one page cover sheet that identifies the title, organization(s), respondent's technical and administrative points of contact - including names, addresses, phone and fax numbers, and email addresses of all co-authors, and clearly indicating its association with RFI-23-02;

2. A substantive, focused, one-half page executive summary;

3. Answers to the above questions including potential research approaches capable of achieving a potential program on this topic limited to 5 pages in minimum 12-point Times

New Roman font, appropriate for single-sided, single-spaced 8.5 by 11-inch paper, with

1-inch margins);

4. A list of citations (any significant claims or reports of success must be accompanied by citations);

5. Optionally, a single overview briefing chart graphically depicting the key ideas;

6. An appendix of critical reference papers or white papers (no more than 3) associated with answers or potential approaches.

Submission Instructions to Respondents:

Responses to this RFI are due no later than 5 p.m., Eastern Time, on 04 August, 2023. All submissions must be electronically submitted to dni-iarpa-rfi-23-02@iarpa.gov as a PDF document. Inquiries to this RFI must be submitted to dni-iarpa-rfi-23-02@iarpa.gov. Do not send questions with proprietary content. No telephone inquiries will be accepted.

Disclaimers and Important Notes:

This is an RFI issued solely for information and planning purposes and does not constitute a solicitation. Respondents are advised that IARPA is under no obligation to acknowledge receipt of the information received or provide feedback to respondents with respect to any information submitted under this RFI.

Responses to this notice are not offers and cannot be accepted by the Government to form a binding contract. Respondents are solely responsible for all expenses associated with responding to this RFI. IARPA will not provide reimbursement for costs incurred in responding to this RFI. It is the respondent's responsibility to ensure that the submitted material has been approved for public release by the information owner.

The Government does not intend to award a contract on the basis of this RFI or to otherwise pay for the information solicited, nor is the Government obligated to issue a solicitation based on responses received. Neither proprietary nor classified concepts or information should be included in the submittal. However, should a respondent wish to submit classified concepts or information, prior coordination must be made with the IARPA Chief of Security. Email the

Primary Point of Contact with a request for coordination with the IARPA Chief of Security.

Input on technical aspects of the responses may be solicited by IARPA from non-Government consultants/experts who are bound by appropriate non-disclosure requirements. Submissions may be reviewed and followed up on by an assigned technical contractor supporting the designated IARPA POC.

Several key laws, enacted over the past three decades, provide general privacy and confidentiality requirements that either directly or indirectly affect all government agencies.

These include The Privacy Act of 1974, The Computer Security Act of 1987, Health Insurance

Portability and Accountability Act of 1996 (HIPAA), US Patriot Act of 2001, and The Confidential

Information Protection and Statistical Efficiency Act of 2002. Under federal law, protected characteristics include race, color, national origin, religion, gender (including pregnancy), disability, age (if the employee is at least 40 years old), and citizenship status. Processing any of these data sets could inadvertently cause discrimination or bias, of these protected characteristics.

Federal laws and regulations that mandate protections for the privacy of citizens are applicable to the use of geospatial data. The Office of Management and Budget (OMB) states in “Circular

A-16 Revised: Coordination of Geographic Information and Related Spatial Data Activities” that geographic and spatial data must not compromise the privacy and the security of personal data about citizens.

Contracting Office Address:

Office of the Director of National Intelligence Intelligence Advanced Research Projects Activity Washington, District of Columbia 20511 United States

Primary Point of Contact:

Ashwini Deshpande

Program Manager - Office of Analysis Research

Intelligence Advanced Research Projects Activity dni-iarpa-rfi-23-02@iarpa.gov

Citations:

1. F. Kong, B. Huang, K. Bradbury and J. M. Malof, "The Synthinel-1 dataset: a collection of high resolution synthetic overhead imagery for building segmentation," 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass, CO, USA, 2020, pp. 1803-1812, doi:

10.1109/WACV45572.2020.9093339.

2. M. F. Reyes, P. D'Angelo and F. Fraundorfer, "SyntCities: A Large Synthetic Remote Sensing

Dataset for Disparity Estimation," in IEEE Journal of Selected Topics in Applied Earth

Observations and Remote Sensing, vol. 15, pp. 10087-10098, 2022, doi:

10.1109/JSTARS.2022.3223937.