N00014-19-S-SN07.pdf

Special Notice N00014-19-S-SN07

June 25, 2019

Special Program Announcement for 2019 Office of Naval Research

Research Opportunity:

Artificial Intelligence/Machine Learning Enabled Capabilities

[in support of the Minerva Innovative Naval Prototype (INP) & the Artificial Intelligence (AI) Task Force]

I. INTRODUCTION

This Special Notice focuses attention on research areas of interest that fall under the Office of Naval Research (ONR) Artificial Intelligence/Machine Learning technology area—this and all other of ONR’s general technology areas are listed on a webpage referred to in Section IIA (“Program Description”) of ONR’s Fiscal Year 2019 (FY19) Long Range Broad Agency Announcement (BAA) for Navy and Marine Corps Science and Technology, N00014-19-S-B001 (and any amendments), at http://www.onr.navy.mil/Contracts-Grants/Funding- Opportunities/Broad-Agency-Announcements.aspx. The submission and evaluation of proposals and the awarding of any grants and contracts for the research areas of interest described in this Special Notice will be made as described in the FY19 Long Range BAA. This Special Notice is not a solicitation.

The purpose of this Special Notice is to focus attention of the scientific community on (1) basic and applied research and development needs for Artificial Intelligence / Machine Learning; (2) identified needs and research interests in turning AI/ML into decision support / decision aiding tools for rapid deployment to Naval Command and Control applications; and (3) the timetable for submission of white papers and proposals addressing the research areas of interest described below.

II. RESEARCH AREAS OF INTEREST

These research areas of interest support the research and development needs of the Minerva Innovative Naval Prototype (INP) program already underway with ONR. Specifically, the context is the Artificial Intelligence/Machine Learning Enabled Capabilities research thrust of the Minerva INP & the Artificial Intelligence (AI) Task Force. ONR seeks new applied research and advanced development concepts that explore and exploit the capabilities of artificial intelligence to create warfighting capabilities related to mission planning and execution, and that will be applicable and training related to Naval command and control as described in NWP 5-01 (DEC 2013), NAVY PLANNING1, (Office of the Chief of Naval Operations, December, 2013).

1 NWP 5-01 NAVY PLANNING can be downloaded from numerous locations on the web including:

http://dnnlgwick.blob.core.windows.net/portals/10/MAWS/5-01_(Dec_2013)_(NWP)- (Promulgated).pdf?sr=b&si=DNNFileManagerPolicy&sig=un5q%2FWUW21Qzq52MmQ7KMfD%2Fh HMdj%2Frp1xJSur5TF58%3D http://www.onr.navy.mil/Contracts-Grants/Funding-Opportunities/Broad-Agency-Announcements.aspx http://www.onr.navy.mil/Contracts-Grants/Funding-Opportunities/Broad-Agency-Announcements.aspx http://dnnlgwick.blob.core.windows.net/portals/10/MAWS/5-01_(Dec_2013)_(NWP)-(Promulgated).pdf?sr=b&si=DNNFileManagerPolicy&sig=un5q%2FWUW21Qzq52MmQ7KMfD%2FhHMdj%2Frp1xJSur5TF58%3D http://dnnlgwick.blob.core.windows.net/portals/10/MAWS/5-01_(Dec_2013)_(NWP)-(Promulgated).pdf?sr=b&si=DNNFileManagerPolicy&sig=un5q%2FWUW21Qzq52MmQ7KMfD%2FhHMdj%2Frp1xJSur5TF58%3D http://dnnlgwick.blob.core.windows.net/portals/10/MAWS/5-01_(Dec_2013)_(NWP)-(Promulgated).pdf?sr=b&si=DNNFileManagerPolicy&sig=un5q%2FWUW21Qzq52MmQ7KMfD%2FhHMdj%2Frp1xJSur5TF58%3D

ONR AI programs will pursue the development and application of novel AI methods to command and control (C2), ISR and training, and in accordance with the mission planning process described in NWP 5-01. Of particular interest are proposals that address Chapters 4 & 5 in NWP 5-01. It is strongly recommended that proposers be familiar with NWP 5-01 and reference appropriate sections from it when describing how the envisaged tools would be used.

Background:

AI refers to the ability of machines to perform tasks that normally require human intelligence such as:

• Recognizing patterns

• Learning from experience

• Drawing conclusions

• Making predictions or inferences

• Taking action (digitally or as the smart software behind autonomous physical systems)

• Understanding goals and objectives

For the research areas of interest highlighted in this Special Notice, the Navy is interested in the application of AI for decision making associated with mission planning, re-planning and execution of Naval operational missions at the tactical and operational command levels. In this context, Artificial Intelligence (AI) is a set of methods that will enable advanced automation, learning, prediction and decision aids to solve complex, dynamic problems. This Special Notice is intended to encourage proposals for novel approaches for leveraging AI/ML for application to advanced C2, Logistics, Intelligence and Training capabilities.

Data: The Navy will have limited data available for the development of AI/ML algorithms.

However, the use of independent data sets for initial development and demonstration of any proposed capabilities is strongly encouraged. Use of data from surrogate tasks, and/or synthesized data to develop initial capabilities is acceptable as part of early phases in a multi-phase (e.g. spiral) development process. As proposed algorithms mature, and after an initial capability demonstration, the Navy will work with developers to get access to appropriate operational data to facilitate further development in an operational context. Proposers should assume access to some operational data will be through limited access (controlled) data portals and will exist in one or more security enclaves. Proposers should anticipate clearance requirements for key personnel accordingly.

Assumptions: Proposers may assume access to data simulation capabilities through a service product. Simulation services will require researchers to configure and run simulation jobs using naval simulation capabilities such as: OneSAF, JSAF and NGTS (mission simulators). It is assumed that any solutions developed in response to this special notice shall be compatible with, if not run on, a service mesh architecture if access to these simulation systems is required. As a result, AI/ML technologies shall need to be delivered as micro-services within any development applications proposed to this topic.

Objective:

ONR is interested in receiving proposals that leverage state of the art AI/ML techniques to enable novel capabilities related to mission planning, as well as: command and control, logistics, intelligence and training for Navy and USMC forces.

This Special Notice does not focus on basic research to develop totally new and/or unproven AI/ML techniques. Any such basic research may be the subject of a separate Special Notice on the Science of AI. Rather, this Special Notice draws attention to research areas of interest that include but are not limited to the following:

Analysis of Factors Affecting Possible Courses of Action

1. Develop and demonstrate the use of natural language processing to enable machines to tailor warfighter support based on commander’s intent (CI) and rules of engagement (ROE) to include:

1.1. Develop tools that can take written text describing ROEs for warfighters and translate them into a machine readable / human interpretable form that may then be processed by machine learning algorithms in the development and assessment of Courses of Action (COAs) for compliance with ROEs & CI.

1.2. Develop tools that can take real time/near real time updates to or clarification of CI / ROE, and appropriately modify the machine representation of CI / ROEs to reflect these updates. These tools would document changes and facilitate curation of CI/ROEs to detect conflicts that might emerge over the course of a mission.

2. Develop AI capabilities that enable identification and orderly examination of all factors that could affect mission execution and expected outcomes.

2.1. Develop a human interpretable dashboard for assessing COAs and recommended COAs status for both the consistency of evolving ROEs, their compliance with human understanding of CI/ROEs, and expectations for mission effectiveness given current CI/ROEs. The dashboard should highlight inconsistencies and missing ROEs required for mission execution. Provide a human interpretable explanation for changes in recommendations developed by AI/ML based algorithms that reflect evolving ROEs.

3. Develop AI capabilities that estimate mission search areas based explicitly stated, or implicitly learned, models of sensor/weapon performance, adversary (enemy) courses of action, and environmental factors that would impact sensor and/or weapon performance.

ECOA Development

4. Formulate learning mechanisms to enable application of knowledge regarding previous courses of action (under similar but potentially different) enemy commander’s intent and enemy rules of engagement. Demonstrate utility of learned relationships to more quickly develop new courses of action under current enemy commander’s intent and enemy rules of engagement.

5. Formulate AI approaches/methods to predict and/or determine most likely, and most dangerous, enemy courses of action (ECOAs).

6. Develop methods to determine significant threat entity patterns of life (PoLs) based upon factual and/or historical behavioral data and/or based on results derived from simulation.

COA Development

7. Develop analytic methods to learn when near-real time meteorological environment observations significantly deviate from forecasts and/or models and do so in a way that will have significant mission impacts.

7.1. Particularly desirable are methods to create advanced climatology products that, while based on historical model data, can be influenced with in-situation observations on a disconnected platform with limited processing capabilities in the absence of new forecast data.

8. Capabilities are desired that can efficiently predict mission measures of effectiveness (MoEs) given tracks from a common tactical / intelligence picture, even with imperfect knowledge.

9. AI approaches are desired that can quickly learn and assess mission-specific objective values (MoVs) based on blue and red force compositions and specific courses of action.

10. Develop AI capabilities to identify critical mission execution factors and specify new critical decisions for a command staff as both red and blue force compositions change due to attrition.

COA/ECOA Assessment & Comparison

11. Develop AI capabilities and approaches that enable large number of courses of action (or warfighter specified decision sequences) to be played against each other using one or more mission objectives and evaluation criteria.

12. Develop AI capabilities that can discover new courses of action and convey to warfighters decision sequences that would include strategies a human staff may never have thought of.

12.1. Algorithmic approaches should explain what is new in the developed courses of action in format that is readily interpretable by warfighters assessing the alternative COAs.

13. Methods that enable alternate decision sequence recommendations based how a mission has played out, including observed decisions and attrition.

14. Of significant interest are algorithmic approaches to COA development that readily enable models trained on one mission context to be quickly re-trained on, and applied to, another.

15. Through observations of an opposing/enemy forces composition of decision sequences, generate methods or algorithms that cluster this information in real-time with past training data sets.

15.1. Algorithms should provide mechanism to ensure optimized model parameters are used and describe the optimizations.

16. Methods enabling end-to-end learning systems to enhance warfighting decision making for specified mission contexts. (Describe the data-to-decision reasoning process across all warfighting areas).

17. Methods to translate a learned optimum decision sequence to the likely decision track of a specific force commander given known biases.

18. Methods to discover the biases of an enemy force commander given deviations between observed decisions and a learned optimum decision sequence to exploit those biases.

19. Relative to a complex decision sequence required by a large force conflict, develop methods to estimate maximum human level performance, maximum possible performance and predicted variances with changes in force composition or mission objectives.

Intelligence Estimation

20. Collection planning and uncertainty estimates to provide a predictive and/or complete Common

Intelligence Picture (CIP)

20.1. Methods that result in better estimates of the current and future locations / behaviors of ground, surface, air and undersea platforms/forces of interest.

20.2. Methods to utilize collection assets to minimize intelligence uncertainty accounting for the relative importance of different information needs and environmental effects.

20.3. Methods to enable collection plans to become more effective over time by learning from past results and by using a rich feature space. Applicable to undersea, surface, air and ground.

20.4. Methods to estimate uncertainty over time given the composition of an opposing force and the composition of own force. Additionally, an estimate of the opposing force’ course of action.

21. Methods to determine enemy relative combat power assessment given possible degradation and incomplete information

22. Methods that incorporate multiple factors (e.g terrain analysis, METOC factors, threat integration etc…) to provide a predictive or more complete CIP

Logistics Estimation

23. Methods that optimize force composability; a service that learns a weighting between survivability, supportability, suitability to perform a task and ability to support information exchange requirements (selectability). Composability can be trained end to end from data or from separately trained services for survivability, supportability, suitability, selectability.

24. Methods to determine Composability of a task or type of mission under temporally constrained timelines. This is managing multiple asset to target pairings continuously in an area of operations.

This includes being able to analyze the four tenants outlined in real time to produce pairings.

25. Methods to determine the Suitability of assets to conduct a task or type of mission. This is determining in near-real time an asset’s “state” which includes status of systems, weapons, geospatial moment, and current tasking (to avoid an asset being removed from a higher priority mission).

26. Methods to determine the Supportability of assets while conducting a task or type of mission. This is determining logistics needed throughout execution, developing objective functions which support logistics based on mission needs and obj.

27. Methods to determine the Selectability of assets while conducting a task or type of mission. This is determining the communications paths needed to maintain asset/weapon interaction under temporally constrained timelines throughout execution.

Communications / Deception

28. Algorithms that can reason about what one side knows or believes about the other based on changes in observed behavior or decision sequences.

Planning & Tasking

29. Methods to translate local observations of an environment to a set of command and control and collection decision sequences, for manned and/or unmanned platforms. Revisit tasking to platform assignments when environmental conditions change.

30. Methods to increase the level of collaboration possible between manned/unmanned platforms relative to finding and fighting an opposition force.

31. Methods to generate optimal blue force (multi-domain, manned, unmanned) laydown given multiple criterion and imperfect opposition force information.

32. Algorithms enabling automated fires planning and engagement processes that classify targets, assign targeting priorities, generate fires plans that synchronize kinetic and non-kinetic effects based on available assets (manned and un-manned), predict outcomes and impact enemy most likely course of action, dynamically update plans, generate mission orders to sub-units, clear fires multi-domain (space, air, surface, sub-surface), and assess battle damage for fires supporting an integrated ground campaign.

Provide for Coordination

33. Optimize platform areas of operation, factoring in the need to prevent blue on blue conflict with mission goal achievement for all domains (undersea, surface, air, ground).

Underlying Support Services

34. Natural language processing to enable machines to better support watch floor information needs.

35. Develop AI capabilities to capture and understand Commander’s Critical Information Requests

(CCIRs) and Priority Information Requests (PIRs), identifying related information, monitoring data streams over time, and alerting warfighters when the information requirements are met.

AI Enablers

36. Develop methods to capture examples of maximum human level performance for a mission in a way that allows future automation.

37. Develop new human machine interface metaphors and schemas to convey the results of AI to Navy warfighters in ways that enable warfighters to (a) understand the pedigree and rationale for the results, so that they can appropriately trust and weigh AI/ML assistance, and (b) quickly and effortlessly harness the results to critical tasks and decisions.

38. AI enabled training environments that test warfighter skill against a thinking opponent whose abilities can be tuned.

39. AI capabilities/methods to compare hierarchical implementations of AI services versus end-to-end implementations.

40. AI capabilities/methods to measure maximum Bayes rule, human level of performance, bias and variance relative to decision making in war gaming or gaming settings.

III. WHITE PAPER SUBMISSION

Although not required, white papers are strongly encouraged for all offerors seeking funding.

Consistent with the FY19 Long Range BAA, each white paper will be evaluated by the Government to determine whether the technology advancement proposed appears to be of value to the current AI/ML development efforts, and ultimately to the Department of the Navy. Initial Government evaluations and feedback will be issued via e-mail notification from the Technical Point of Contact. The initial white paper appraisal is intended to give entities a sense of whether their concepts are likely to be funded, and therefore whether to invest resources in developing a Detailed Full Proposal.

Detailed Full Proposals (Technical and Cost volumes) will be subsequently encouraged from those Offerors whose proposed technologies have been identified through the above referenced white papers as being of “particular value” to the Government under the FY19 Long Range BAA. However, any such encouragement does not assure a subsequent award. Full Proposals may also be submitted by any offeror whose white paper was not identified as being of particular value to the Government or any offeror who did not submit a white paper.

For white papers that propose efforts that are considered of particular value to the Navy but either exceed available budgets or contain certain tasks or applications that are not desired by the Navy, ONR may suggest a full proposal with reduced effort to fit within expected available budgets or as a modified effort that refocuses the tasks or application of the technology to maximize the benefit to the Navy.

For detailed instructions on submitting a white paper, refer to Section II.D.2.d. of ONR’s Long Range BAA (N00014-19-S-B001and any amendments).

The following supplemental instructions apply to white papers submitted for Research Areas addressed in this Special Notice.

The cover page of the white papers should be labeled “White Paper for ONR 2019 Research Opportunity: Artificial Intelligence/Machine Learning Enabled Capabilities” and include the following information: title of the proposed effort, technical point of contact, telephone number, fax numbers, and e-mail address for Principal Investigator(s).

The 5-page body of the white paper should include the following information:

(1) Principal Investigator Name & Affiliation;

(2) Relevance of the proposed effort to the research areas described in Section II;

(3) Technical objective of the proposed effort;

(4) Technical approach that will be pursued to meet the objective;

(5) A summary of recent relevant technical breakthroughs; and

(6) A funding plan showing requested funding per fiscal year.

A resume of the principal investigator(s), not to exceed 1 page, should also be included after the 5-page body of the white paper.

To ensure full, timely consideration for funding, white papers should be submitted no later than 15 July 2019. White papers received after that date will be considered as time and availability of funding permit.

The planned date for completing the review of white papers is 7 August 2019.

https://www.onr.navy.mil/work-with-us/funding-opportunities/announcements

IV. FULL PROPOSAL SUBMISSION AND AWARD INFORMATION

Full proposals should be submitted under the FY19 Long Range BAA (N00014-19-S-B001 and any amendments) by 6 Sep 2019. Full Proposals received after that date will be considered as time and availability of funding permit.

For detailed instructions on submitting a full proposal, refer to Section II.D.2.e. of ONR’s Long Range BAA.

The following supplemental instructions apply to proposals submitted for Research Areas addressed in this Special Notice.

ONR anticipates that both grants and contracts will be issued for this effort.

For grants, the following information must be completed in the SF 424 to ensure that the application is directed to the correct individual for review: Block 4a, Federal Identifier: Enter N00014; Block 4b, Agency Routing Number, Enter the three (3) digit Program Office Code

(341) and the Program Officer’s name, last name first, in brackets (“Kruger, Martin”). All attachments to the application should also include this identifier to ensure the proposal and its attachments are received by the appropriate Program Office.

ONR plans to fund five to fifteen individual awards with a value of 1 million to 3 million per year, using research funds. However, lower and higher cost proposals will be considered.

The period of performance for projects may be from two to four years.

Although ONR expects the above described program plan to be executed, ONR reserves the right to make changes due to availability of funds, and/or guidance of senior leadership.

Funding decisions are expected to be made by 4 October 2019. Selected projects will have an estimated award date of 7 Feb 2020.

V. SIGNIFICANT DATES AND TIMES

Event Date Time

Recommended White Paper Submission Date*

15 July 2019 COB

Notification of White Paper Valuation* 7 August 2019 COB Recommended Full Proposal Submission 6 September 2019 COB Notification of Selection: Full Proposals

4 October 2019 COB

Awards * 7 Feb 2020 COB

Note: * These are approximate dates.

https://www.onr.navy.mil/work-with-us/funding-opportunities/announcements

VI. POINTS OF CONTACT

In addition to the points of contact listed in N00014-19-S-B001, the specific points of contact for this announcement are listed below:

Technical Points of Contact:

Martin Kruger Martin.kruger1@navy.mil

Business Point of Contact:

Phillip Lee phillip.m.lee2@navy.mil

VII. ADDRESS FOR THE SUBMISSION OF WHITE PAPERS AND FULL

PROPOSALS FOR CONTRACTS

White papers should be submitted electronically to the program technical points of contact, Martin Kruger. Files exceeding 10MB in size should not be emailed, but instead transmitted via a file transfer service, for example AMRDEC Safesite, https://safe.amrdec.army.mil, or mailed on DCROM or DVD.

If an offeror chooses, a DVD or CD-ROM of the Full Proposal including all supporting documentation can be sent to the Office of Naval Research at the following address:

Primary Point of Contact Secondary Point of Contact Office of Naval Research

Attn: Martin Kruger ONR Department Code 34

875 North Randolph Street – Suite 1160 Arlington, VA 22203-1995

Office of Naval Research Attn: Robert Palisin

ONR Department Code 311 875 North Randolph Street – Suite 1106

Arlington, VA 22203-1995

VIII. SUBMISSION OF QUESTIONS

Any questions regarding this announcement must be provided to the Technical Points of Contact and/or the Business Point of Contact listed above. All questions shall be submitted in writing by electronic mail.

Answers to questions submitted in response to this Special Notice will be addressed in the form of an Amendment and will be posted to the following web pages:

• Federal Business Opportunities (FEDBIZOPPS) Webpage – https://www.fbo.gov/

• Grants.gov Webpage – http://www.grants.gov/

• ONR Special Notice Webpage - http://www.onr.navy.mil/Contracts-Grants/Funding-

Opportunities/Special-Notices.aspx https://safe.amrdec.army.mil/ https://www.fbo.gov/ http://www.grants.gov/ http://www.onr.navy.mil/Contracts-Grants/Funding-Opportunities/Special-Notices.aspx http://www.onr.navy.mil/Contracts-Grants/Funding-Opportunities/Special-Notices.aspx

Questions regarding White Papers or Full Proposals should be submitted NLT two weeks before the dates recommended for receipt of White Papers and/or Full Proposals. Questions after this date may not be answered.