BAA_Questions_Amendment_0002.docx

BAA Questions and Answers Expeditionary Intelligence, Surveillance, and Reconnaissance Science and Technology

BAA#: N00014-18-S-B006

1. Would this BAA include research of Computer Vision technologies that can provide input into the framework discussed in the solicitation?

A. The use of computer vision algorithms to interpret patterns in diverse data would be responsive.

2. How much user involvement would you anticipate when training an AI or machine learning system? Can a system work interactively with a user to transfer knowledge or generate training data, or should it be fully automated?

A. If by user you mean a warfighter, any machine learning from humans needs to be accomplished unobtrusively.

3. When discussing “patterns-of-life”, do you have specific data sources or representations in mind (e.g. features/categories, hierarchy, temporal, geospatial, etc.)?

A. Patterns of like for entities that represent an enemy order of battle are of interest. Those entities have properties as well as space/time movement patterns.

4. Several topics mention generation of training data or otherwise imply classification. What is the sensing domain of primary interest? Are we talking EO/IR or acoustic? Is aerial aerial/drone perception of interest or is this principally focused on underwater?

A. All modalities. Data from aerial sensors are of greater interest.

5. In general, are you ok with utilizing public datasets in support of ML training, or do you want us to focus on generating entirely new data from scratch?

A. Any available data set that that improves the performance of an agent should be used.

6. There is a lot of mention of agents, but a number of the areas of interest could be tackled using non-agent-based AI approaches. Is the TPOC only interested in agent-based approaches?

A. No. Any approach that uses AI/ML to enable a capability is potentially of interest.

7. What about approaches that use human authored models of adversary behaviors, with the longer-term goal of machine learning approaches to model creation? Or is the machine learning element critical in earlier phases of the research?

A. Offerings should have a strong ML component.

8. Regarding the topic related to battle management aids, do you have any particular success metrics in mind for evaluating battle management aids?

A. Any aids that enhance force protection, mission effectiveness or mission execution efficiency across collection and C2 are of interest.

9. Question on the application of GANs. Is ONR interested in data of a particular modality with regard to this focus area? What type of data may be most relevant for demonstrating the applicability of an approach to ONR’s concerns?

A. The use of GANS to generate labeled data to train any of the services described is of interest.

10. Many of the areas of interest involve augmenting, enhancing or predicting within a common tactical picture. Are there particular standards, existing schemas, repositories or APIs that ONR would like proposers to adhere to?

A. This is dependent on the service being provided and whether the Marine Corps or Navy would be the transition partner.

11. In terms of inputs to the tactical picture, what communications or sensor data is within scope (e.g. on-ship communications, Blue force audio/chat, SIGINT, ELINT)

A. All. Note improving data access is not within scope of the BAA.

12. Are there any envisioned size/weight/power constraints on a fieldable prototype at this stage in your development cycle?

A. No. The constraints would vary with level of command.

13. And one procedural question, are you looking for resumes of key personnel in the whitepaper? (My assumption is that is more an expectation for the proposal, but wanted to clarify)

A. Evaluators will consider the qualifications, capabilities and related experience of the offeror, proposed Principal Investigator (PI), team leader, and key personnel.

14. Can an offeror submit multiple white papers?

A. Yes

15. What is the application domain? Is computer vision of interest? Are only EO image data types of interest or are the SAR and Radar inputs also considered?

A. Computer vision algorithms may be relevant. Analysis of embeddings is more important than EO images.

16. What is the exit criterion or metric of success? Detection Accuracy vs. number of synthetic training data samples (for augmentation)?

A. The more robust the service is the better. Different services may have different metrics for operational relevance

17. What are the datasets and targets of interest? Are Battleships/Buildings/Port structures of interest?

A. Any object in a military order of battle is of interest.

18. What is the target classifier of interest? Is YOLO an acceptable classifier?

A. Classifier selection is up to offerors.

19. Does the target classifier detect targets, or classify target types, or label more refined features?

A. Threshold is battlespace situational awareness/understanding and decision support.

20. What is the size of available training data? What types of labeled datasets are available?

A. Data sets generated by military simulations are available.

21. Is the target AI system fully automated? Is it possible to have human in the loop?

A. The threshold system would be human on the loop.

22. Do you anticipate that domain knowledge could available and be leveraged to generate synthetic data?

A. Potentially, dependent on what domain knowledge is desired.

23. To train AI with less data, do we have to use generative adversarial networks based augmentation? Are other few-shot learning methods also in scope?

A. Methods choses are up to offerors

24. Are you interested in only training data augmentation or also training data adaptation (domain adaptation, e.g., from day time images to night time images)?

A. Question is not relevant to the BAA

25. Is there any time and computational complexity constraint that we need to take into account?

A. Most military decision support systems need to run in near real-time to be useful

26. What simulation environments are available to train an agent? If the Navy can’t provide such an environment, can we use public environments?

A. JSAF/OneSAF can be made available

27. Is there a target transition program or are there potential Navy end-products for integration at the end of the project?

A. Navy and USMC command and control programs of record

28. Who will be the end-user? Will it be the researcher or analyst using the Docker to augment a dataset or a system (target recognition system) that is trained with augmented dataset?

A. Different services will have different end users

29. Should the ROM include the entire period of performance (potentially 3 years) or just the first year?

A. Entire period of performance

30. In the white paper do we specify that we are applying for BA2 or BA3

A. Not required

31. Should the white paper provide a preliminary project schedule (e.g., Gantt Chart)?

A. Potentially useful but not required

32. Given existing relationships with a number of individual organizations and academic institutions that may also be submitting white papers, is there the potential for teams to be formed after white paper submission?

A. The government will not form such teams.

33. Will there be preference for products with existing ML/AI capabilities, over products with potential future integration?

A. Both are responsive.

34. Are there preferred qualifications for a successful PI? Are experiences and capabilities valued over educational qualifications?

A. Both experience and education qualifications that support the researcher’s ability to be successful are important

35. Is novel visualization of data retrieved from agents a preferred capability?

A. That is considered to be in-scope

36. Given existing capabilities for developing training environments and identifying training points, how may we evaluate whether said capabilities satisfy criteria for success, or where improvements can be made to meet said criteria? What are desirable metrics for successful solutions?

A. That is largely the responsibility of an offeror to determine. Any developed capability should have operational or potential operational relevance.

37. In the last bullet of the Naval areas of interest, can you clarify if the “little or no communications” limits the communication between the commander/operator and the agents, between the agents themselves, or both?

A. Between the platforms/sensors