SCEPTER_QA_01282022.pdf

SCEPTER

HR001122S0013

Frequently Asked Questions

Last Updated: 1/28/2022

Q1. Will it be possible to get access to a video of the session via SAFE later, if we cannot get in, or a copy of the slides?

A1. No. Slides will not be provided. No video will be provided.

Q2. About identifying the best courses of actions, can we use human in the loop learning to select courses of actions? or can it be a manually selected?

A2. Yes. The user interface (UI) and user experience (UX) should be thoughtfully considered.

Q3. Will you be sharing contact information for teaming purposes?

A3. Yes, the industry day registration list will be shared for the purposes of teaming.

Q4. Are there any KPIs that should be addressed other than those that were briefed today? Can you provide KPIs for each spiral?

A4. Performance metrics are very important to the program and should be specifically addressed in proposals. Additional metrics may be proposed especially if they more clearly demonstrate progress against the program goals.

Q5. Can you provide realistic scenarios for FA1 so we can formulate our proposed architecture for this effort?

A5. Proposers may use historical scenarios or proposer defined scenarios for the purposes of proposing.

Q6. Do you have a preferred simulator that performers should select? You listed several options.

A6. No. Proposers are encouraged to select the simulation environment that they feel are most likely to achieve the program metrics.

Q7. Can you say what the scenario in Phase 1 will be?

A7. See response to Question #5.

Q8. Could you clarify what is a "unit" for purposes of FA2 metrics? A military formation is a unit with potentially many entities. Are you measuring directly controlled formations or total # of entities?

A8. Solutions that can accommodate flexible unit definition to explore various scenario resolutions are desirable. By way of example, in some scenarios a unit may represent a single aircraft. In other scenarios a squadron or a wing may be represented as a single ‘unit’.

Q9. In the evaluation of the COAs, there is a qualitative eval of whether the COAs are human-interpretable (e.g. "looks like" an ambush). There is a risk when training in sims that the AI has learned to exploit something that is simulation-specific (e.g. breaking physics). Is the role of the trusted sims (e.g. OneSAF) to provide an environment where we can safely assume nothing like that is possible, or should we include metrics/justification in the evaluation portion of our proposal to validate/verify that the AI has learned a genuine, generalizable strategy vs. sim-specific strategy?

A9. We recognize that trusted, “high-fidelity” simulators are not perfect. Proposers should demonstrate an understanding of the strengths and weaknesses of the simulator proposed. Proposers may propose improvements as necessary to overcome any limitations that prevent the SCEPTER validation as described in the BAA.

Q10. Assume simulations will incorporate denied/noisy/partial information and that ISR will be important aspect (along with comms and logistics). Correct?

A10. Correct.

Q11. Section 1.1.5 (last paragraph) indicates that proposals should define a UI/UX. Summary bullets indicate that a Phase 1 Objective is to design a UI/UX. Please clarify what constitutes a sufficiently detailed design to satisfy deliverable requirements? Please verify that building the UI/UX is not a

Phase 1 goal. When and by whom will the UI/UX be built?

A11. The BAA outlines program metrics and specifically does not define solution requirements. The

Phase 1 objectives include executing the SCEPTER capability. The UI/UX is considered part of the

SCEPTER capability.