HR001119S0035-07.pdf

Small Business Innovation Research (SBIR) and Small Business Technology Transfer

(STTR)

Opportunity Announcement

HR001119S0035-07

CHAracterization of Intrinsic Novelty (CHAIN)

Which program will fund this topic?

SBIR

What type of proposals will be accepted?

Phase I Only

Technology Area(s): Human Systems, Information Systems Technology DARPA Program: Science of Artificial Intelligence and Learning for Open-world Novelty

(SAIL-ON)

I. INTRODUCTION

The Defense Advanced Research Projects Agency (DARPA) Small Business Programs Office (SBPO) is issuing an SBIR/STTR Opportunity (SBO) inviting submissions of innovative research concepts in the technical domains of Human Systems and Information Systems Technology. In particular, DARPA is interested in understanding the feasibility of CHAracterization of Intrinsic Novelty (CHAIN) and its use as the basis for evaluating the robustness to novelty of AI applications.

This SBO is issued under the Broad Agency Announcement (BAA) for SBIR/STTR, HR001119S0035. All proposals in response to the technical area(s) described herein will be submitted in accordance with the instructions provided under HR001119S0035, found here:

https://www.fbo.gov/spg/ODA/DARPA/CMO/HR001119S0035/listing.html

a. Eligibility The eligibility requirements for the SBIR/STTR programs are unique and do not correspond to those of other small business programs. Please refer to Section 3.1, Eligible Applicants, of HR001119S0035 for full eligibility requirements.

b. Anticipated Structure/Award Information Please refer to Section 1, Funding Opportunity Description provided in HR001119S0035 for detailed information regarding SBIR/STTR phase structure and flexibility.

For this SBO, DARPA will accept Phase I proposals only, for cost of up to $225,000 for a 6 month period of performance. Proposers awarded a Phase I contract will be eligible to submit a proposal for Phase II, for a cost up to $1,250,000. This includes an 18 month base period not to exceed a cost of $1,000,000 and a separately priced option of up to $250,000 for participation in the DARPA Entrepreneurial Investigator Initiative.

Proposers should refer to Section 4, Application and Submission Information, of HR001119S0035 for detailed proposal preparation instructions. Proposals that do not https://www.fbo.gov/spg/ODA/DARPA/CMO/HR001119S0035/listing.html comply with the requirements detailed in HR001119S0035 and the research objectives of this SBO are considered non-conforming and, therefore, are not evaluated nor considered for award.

Phase I proposals shall not exceed 20 pages. Phase I commercialization strategy shall not exceed 5 pages. This should be the last section of the technical volume and will not count against the 20-page limit. Please refer to Appendix A, Phase I Proposal Instructions, of HR001119S0035 for detailed instructions on proposal preparation.

c. Evaluation of Proposals Section 5, Evaluation of Proposals, in HR001119S0035 provides detailed information on proposal evaluation and the selection process for this SBO.

d. Due Date/Time Full proposal packages (Proposal Cover Sheet, Technical Volume, Price/Cost Volume inclusive of supporting documentation, and Company Commercialization Report) must be submitted via the DoD SBIR/STTR Proposal Submission website per the instructions outlined in HR001119S0035 no later than August 6, 2019 at 2:00 p.m. ET.

II. TOPIC OVERVIEW

a. Objective

Define, develop, and demonstrate scientifically-based, practical, and effective verification and validation (V&V) techniques for novelty-robust AI applications in a selected domain.

b. Description Military uses of AI are anticipated to be widespread. Military operations are typically characterized by novel situations, which we define as those that violate implicit or explicit assumptions of an AI application. Military AI applications will need to be robust to novelty, and this characteristic requires scientifically-based verification and validation.

This effort will provide a comprehensive scientific basis of novelty leading to the definition, development, and demonstration of innovative, appropriate and useful V&V techniques for novelty-robust military AI applications, as well as for commercial AI applications with similar characteristics. The approaches must be applicable across multiple application domains and degrees of autonomy (e.g., fully-autonomous, semi-autonomous, human-machine partnership, etc.).

Current research primarily describes novelty in terms of how human or other intelligent agents recognize and/or react, rather than in terms inherent to the novelty characteristics themselves. Novelty characteristics to consider include domain and task characteristics, types of novelty, measures and measurements of novelty, novelty generation methods, dimensions of novelty, theoretical bases for characterizations of novelty, and resolved and open issues regarding novelty. V&V techniques that are applicable both to military and commercial applications are preferred; however, unique requirements of military applications should be identified and considered when necessary. The proposed approaches should address not only technical methods for evaluation, but also consider business, operational, user, and environmental aspects, so the researched V&V methods can be adopted and applied in practice. The proposed techniques should be considered in the context of a business plan to offer a product or service to DoD and commercial customers.

The Phase 1 objective is to provide the scientific basis for and to identify feasible and scientifically sound V&V techniques. In Phase 2, the objective is to develop, demonstrate, and evaluate V&V techniques established based on the research conducted in Phase 1. One domain, to be proposed, will be used in order to demonstrate the applicability of the methods and techniques as a mechanism for assessing robustness to novelty when evaluating AI systems.

In Phase 1 performers will have two deliverables: (1) a report detailing the research into novelty characterization and (2) a plan for practical validation of the novelty robustness of AI systems. In Phase 2 performers will apply the identified V&V techniques to a selected domain and assess the utility of these techniques from multiple perspectives, including domain/task generality and applicability, ease of implementation and use, understandability and trustworthiness, and ability to quantify results. Deliverables will include a demonstration of the proposed V&V techniques, a report with an analysis of the appropriateness, utility, applicability, benefits, limitations, etc. of the techniques employed, and a business plan for extending and applying these techniques to a range of DoD and commercial AI applications.

c. Phase I Provide the scientific basis necessary to identify, propose, and analyze appropriate techniques for V&V of AI applications with respect to their ability to function appropriately and effectively in novel situations. This should be based on existing scientific research into inherent characterizations of novelty, as distinct from how people or other intelligent agents react to novelty, and should include (1) a comparison of qualitative and quantitative characterizations and measures of novelty intrinsic to a domain and task characteristics; (2) an explanation of the theoretical basis for such characterizations; (3) identification of well-studied and still unresolved issues for which further research is needed. This knowledge should provide insights for the development of V&V techniques for novelty-robust AI applications. Those insights and V&V techniques should then be used in the development of a business plan for the general evaluation of AI systems. End of Phase I should result in a comprehensive report on the scientific underpinnings and plan for practical deployment of the proposed V&V techniques for novelty-robust AI systems.

Phase I fixed payable milestones for this program should include:

Month 1: Research Plan, which is an updated, detailed and expanded version of the information provided in the proposal. The Research Plan should identify the information to be obtained, the method for obtaining it, mechanisms for capturing, documenting, and validating the information received, and any anticipated follow up actions that may be needed.

Month 3: Research report, which details existing research relevant to novelty characterization.

Month 4: Draft Feasibility Analysis Report addressing all the items identified in the Phase 1 objective (above), and any other factors identified as relevant by the performer. In particular, the report shall document the need for researching, developing, customizing scientific techniques as needed to enable the solution.

Presentation to PM of results.

Month 6: Final Feasibility Analysis Report addressing all the items identified in the Phase 1 objective (above), and any other factors identified as relevant by the performer. In particular, the report shall document the results of researching and developing qualitative and quantitative characterizations of novelty and proposed methods for validation and verification of novelty-robust AI applications in military and commercial domains.

PI meeting presentation material (every 6 months, to be synchronized with other program PI meeting schedules).

d. Phase II Implement V&V plan for incorporating novelty into evaluation of AI systems in a selected domain and analyze utility of the V&V techniques from multiple perspectives, resulting in a business plan for extension and application of these techniques to multiple DoD and commercial applications. The implementation should occur in a to-be-proposed domain, which is selected to be broadly representative of a set of AI applications that are anticipated to be developed for widespread use in DoD. It should result in a demonstration of the developed V&V techniques and a qualitative and quantitative assessment of their coverage, generality, applicability, utility, practicality, understandability, ease of use, and benefits. The results of this assessment should be used to develop a business plan for extension and application of these techniques to a range of potential DoD and commercial customers.

Phase II fixed payable milestones for this program should include:

Month 7: Technique and Demonstration plan that identifies specific V&V methods and capabilities to be selected, enhanced, modified, developed, and applied and identifies a specific AI application for demonstration and assessment of these techniques. The report should explain how novelty will be systematically introduced into the domain and how this novelty can be characterized as the basis for evaluation of the robustness of the AI application.

Month 12: PI meeting presentation material, including demonstration of progress to date, PowerPoint presentations of accomplishments and plans. (This date is not firm;

it may be adjusted to be synchronized with PI meetings of related DARPA programs;

so this date should not be considered a payable milestone, rather, only a formal deliverable.)

Month 15: Demonstration Logistics Plan that identifies logistical requirements for the pilot demonstration.

Month 18: Initial demonstration of V&V techniques for novelty-robust AI applications to DARPA.

Month 19: Report on lessons-learned from initial demonstration regarding needs and plans for additions and modifications needed for V&V of a novelty-robust AI methods.

Month 22: Final Demonstration of application of V&V business plan to DARPA, expanding the formalized metrics in a systematic way.

Month 23/24: Draft and Final reports, comprising a business plan for offering a product and/or service to commercial and DoD customers for V&V of AI applications demonstrating robustness to novelty; i.e., the ability to degrade gracefully and/or act appropriately or safely even when implicit or explicit assumptions are violated. The reports must address domain and task characteristics, qualitative and quantitative metrics, and technical and human aspects.

PI meeting presentation material (every 6 months, to be synchronized with other program PI meeting schedules).

e. Dual Use Applications (Phase III) Develop a dual-use service and/or software product (the service could be offered with an accompanying software enabler) to execute V&V of AI applications in domains that require regulatory approvals to operate. This service could be offered to automobile companies to test the AI in driverless cars for approval to sell for use on public roads, to medical diagnosis companies (or insurance companies) that want to use AI for treatment decisions, for consumer facing financial companies using AI for credit decisions, etc. It could also be used for military applications of AI to verify performance specs or to verify that unacceptable behaviors will not occur in unknown conditions.

f. References http://lvl.info.ucl.ac.be/pmwiki/uploads/Publications/VerificationAndValidationAndArtif icialIntelligence/aivvis-aic.pdf https://arxiv.org/pdf/1808.08444.pdf www.nellybencomo.me/publications/2014/BencomoNIER2014.pdf

g. Keywords Verification, Validation, Artificial Intelligence, Novelty, Human Systems, Metrics, Formalism

III. SUBMISSION OF QUESTIONS

DARPA intends to use electronic mail for all correspondence regarding this announcement.

Questions related to the technical aspect of the research objectives and awards specifically related to this SBO should be emailed to HR001119S0035@darpa.mil. Please include BAA number HR001119S0035-07 in the subject line. All questions must be in English and must include the name, email address, and the telephone number of a point of contact.

DARPA will attempt to answer questions in a timely manner; however, questions submitted within 7 days of the proposal due date listed herein may not be answered. DARPA will post a consolidated Frequently Asked Questions (FAQ) document. To access the posting go to:

http://www.darpa.mil/work-with-us/opportunities. Under the HR001119S0035-07 summary will be a link to the FAQ. The list will be updated on an ongoing basis until one week prior to the proposal due date.

http://lvl.info.ucl.ac.be/pmwiki/uploads/Publications/VerificationAndValidationAndArtificialIntelligence/aivvis-aic.pdf http://lvl.info.ucl.ac.be/pmwiki/uploads/Publications/VerificationAndValidationAndArtificialIntelligence/aivvis-aic.pdf https://arxiv.org/pdf/1808.08444.pdf http://www.nellybencomo.me/publications/2014/BencomoNIER2014.pdf

In addition to the FAQ specific to this SBO, proposers should also review the SBIR/STTR General FAQ list at: http://www.darpa.mil/work-with-us/opportunities. Under the HR001119S0035 summary will be a link to the general FAQ.

Technical support for the DoD SBIR/STTR Proposal Submission website is available Monday through Friday, 9:00 a.m. – 6:00 p.m. ET. Requests for technical support must be emailed to sbirhelp@bytecubed.com with a copy to sbir@darpa.mil.