HR001123S0025_FAQ.pdf

Faithful Integrated Reverse-engineering and Exploitation (FIRE)

HR001123S0025

Frequently Asked Questions (FAQs)

Question 1: For evaluation of CPVA tools, what additional artifacts/information (e.g. source code or system spec.), in addition to the subject CPS, will you provide to the performers?

Answer 1: We can’t comment on information that might be specific to approaches. Recommend the proposer please describe assumptions, risks, and mitigations in the abstract and/or proposal.

Question 2: Can you tell us which processor architectures you are interested in?

Answer 2: Please see Question 1.

Question 3: What kind of data will be provided for modeling purposes?

Answer 3: Please see Question 1.

Question 4: Test article composition will make a big difference in what we propose. A drone w/ an FPGA in it vs. a smart meter w/ a CPU are very different. When will we learn what the test will be?

Answer 4: Please see Question 1.

Question 5: Are we operating with design info like: manuals, schematics, source code, encryption keys?

Answer 5: Please see Question 1.

Question 6: FPGA tools and techniques like determining encryption key and extracting netlists from bitstreams have been invented and re-invented on numerous programs. For efficiency, may we; 1) assume we will get the netlist for any included FPGA or 2) assume we have access to such tools provided by the program?

Answer 6: Please see Question 1.

Question 7: CPVA may be highly domain-specific (e.g. drones vs. ICS). Do you expect each proposal to propose generic CPVA tools applicable to a wide range of CPS, or it’s OK to propose domain-specific

CPVA tools (e.g. for drones only)?

Answer 7: Please see Question 1.

Question 8: If a TA5 performer has ideas for a “design for analysis” CPS system that has a design goal, making FIRE evaluation easier, is R&D of that sort in scope?

Answer 8: Please see Question 1.

Question 9: Are you expecting a solution across FPGA, DSP, PLC, CPU or should we propose with deep expertise in one family?

Answer 9: Please see Question 1 and Section 1.B.1 “TA1 Modeling: Scaling Complexity in Models while Maintaining Accuracy “in the BAA.

Question 10: Would you be open to AI/ML as being part of our approach? (if it makes sense of course)

Answer 10: We are open to approaches that meets the program goals and metrics. Please see

Section 1.B “Program Description” of the BAA for the program goals and metrics. Also see

Question 1.

Frequently Asked Questions (FAQs)

Question 11: Are stochastic countermeasures and ML-based approaches in scope even if they may not eliminate an attack vector completely?

Answer 11: Please see Question 1.

Question 12: What protocols do we need to support?

Answer 12: Please see Question 1.

Question 13: What communications bus do we need to support? CAN? 1553? Ethernet?

Answer 13: Please see Question 1.

Question 14: Which TA is responsible for vulnerability discovery and patch generation (seems cross-cutting over TA1, TA2, TA4)?

Answer 14: Please see Section 1.B “Program Description” of the BAA for specific descriptions.

Question 15: The BAA describes a preference for integrated TA1-TA4 proposals. Does this imply the expected award for an integrated TA1-TA4 solution could be multiple times an individual TA award?

Answer 15: There is no implication. Please see Section V.A “Evaluation Criteria” in the BAA for evaluation criteria.

Question 16: With GOMAC being next week can the abstract deadline by extended by 1 week?

Answer 16: No, we cannot make special arrangements to accommodate specific proposers.

Question 17: A typical DARPA BAA has a 4 week period between BAA release and abstract submission.

FIRE only has 2 weeks and most of this community will be at GOMAC next week. Can the abstract due date be pushed back 1-2 weeks?

Answer 17: Please see Question 16.

Question 18: Can DARPA extend the abstract deadline by a week? The abstract for one TA is 4 pages if multiple Tas, it requires more pages. This is a complex program two-week timeline is too short.

Answer 18: Please see Question 16.

Question 19: Does the government plan to downselect performers even if they meet the metrics for each phase?

Answer 19: We will not speculate on future decisions.

Question 20: Does one need a clearance to participate and bid?

Answer 20: Please see Section 3.B.3 “Ability to support classified development” of the BAA for eligibility information.

Question 21: Will the lightning session slides be published?

Answer 21: No.

Frequently Asked Questions (FAQs)

Question 22: Will the CPSs include both OT and IT networks. Example: the medical pump includes the nurse’s station computer.

Answer 22: Please see Question 1.

Question 23: Would a TA-5 system need commercial equipment, or could synthetic but representative equipment be used?

Answer 23: Please see Question 1.

Question 24: Do we have to submit an abstract?

Answer 24: No. Abstracts can be used to obtain initial feedback which might inform proposal writing decisions and whether to continue with proposal submission.

Question 25: Will submitting an subtract improve our chances of getting a contract?

Answer 25: Please see Answer 24.

Question 26: can you clarify the scope of a software component?

– Image level “bitstream” “binary”

– library e.g. crypto library w/ multiple algorithms

– instance level e.g. single crypto algo from a library

Answer 26: A component is defined in Section 1.A “Background” of the BAA. A single “binary” may contain one or more components.

Question 27: can you clarify what constitutes a hardware component? When you say “up to 100” are you describing processing/sensing components or does that include passive (resistors, capacitors).

Answer 27: A component is defined in Section 1.A “Background” of the BAA. We can’t speculate on the purpose of resistor or capacitors for example.

Question 28: Is there any human-in-the-loop or the whole analysis should be automated? If there is a human, what competencies should one possess: SW/HW/RE engineering?

Answer 28: How you choose to use your 1-month metric period is up to your technical approach. Please describe assumptions, risks, and mitigations in the abstract and/or proposal.

Also see Question 1.

Question 29: Is the human operators and their perception of system health/status in scope for attack path modelling?

Answer 29: Please see Question 28.

Question 30: Other than “one month,” what resource utilization targets do you envision?

- Total staff-months during that month window

- Total reusable + non-reusable infrastructure costs (compute, instrumentation, space, power, etc.)

Frequently Asked Questions (FAQs)

Answer 30: Please see Question 28.

Question 31: Are there metrics for patch effects on benign/intend system behavior?

Answer 31: Please refer to the BAA on proposing patch metrics.

Question 32: Is survivability vs. complete exploit neutralization a valid FIRE outcome?

Answer 32: Please see Question 31.

Question 33: What patches are in scope? Examples: re-image firmware, update O/S or control s/w, reprogram FPGA, replace sensor, add ASIC or device to control board, re-architecture system to isolate unpatchable component, etc.

Answer 33: Please see Questions 31 and 1.

Question 34: Are patches that raise the bar for future attacks in scope? E.g.: reduce likelihood of success, raise the risk of attributability/detectability, raise the likelihood of an alarm presented to a human operator, etc.

Answer 34: Please see Questions 31 and 1

Question 35: Some vulnerabilities can be hidden by other vulnerabilities. Detecting the second can only be done if the first is corrected. If patching is not measured, how will DARPA handle these scenarios.

Answer 35: Please see Question 31.

Question 36: Is a goal of this program automated exploitation and patching, or should we just be focusing on the environmental setup to be able to conduct this analysis?

Answer 36: Please refer to the program description and metrics in Section 1.B.” Program

Description” of the BAA.

Question 37: Can you explain more about TA3? What are you looking for? Why wouldn’t TA4 do the preparation for integration purposes?

Answer 37: Please refer to Section 1.B.” Program Description” of the BAA for the description.

Question 38: What is the motivation/rationale behind the 1 month schedule?

Answer 38: Please refer to Section 1.B.” Program Description” of the BAA for the program metrics description.

Question 39: TA5 is charged with building the test + evaluation models. Are TA1-TA3/TA4 performers expected or allowed to define the targets they use for the research phase?

Answer 39: Please refer to the BAA for IV&V descriptions. Also see Question 1.

Question 40: Which TA is responsible for defining a patch? That’s not an analysis problem.

Answer 40: Please see Question 31.

Question 41: How will the expected behavior(s) of an exploit be specified? E.g., A drone losing altitude by 30m will crash if it is flying at 20m. Is the exploit crash or the loss of altitude?

Frequently Asked Questions (FAQs)

Answer 41: Exploits are defined in Section 1.A. “Background” of the BAA.

Question 42: Isn’t defining relevant unexpected behavior part of the problem statement?

Answer 42: Please see the definitions of vulnerabilities and exploits within Section 1.a.

“Background” of the BAA.

Question 43: Is there a “weight” value to any TA within a combined proposal?

Answer 43: No. Evaluation criteria can be found in Section V.A “Evaluation Criteria” of the BAA.

Question 44: What are the expected interactions between TA3 and TA5? Will there be models provided?

How sophisticated will they be?

Answer 44: Please see Questions 1 and 39.

Question 45: Are we allowed to propose self-defined interim metrics to show progress for our solution?

Answer 45: Yes.

Question 46: Are we allowed to propose self-defined interim CPS to show progress for our solution?

Answer 46: Yes. Also see question 45.

Question 47: Can you please clarify what quantity of the systems will be needed for each specific deliverable for TA5?

Answer 47: This will be clarified in Section 1.D. “Schedule/Milestones” of Amendment 01 of the

BAA.

Question 48: Can you please clarify how long each of the QPRs should be for costing purposes?

Answer 48: This will be clarified in Section 1.D. “Schedule/Milestones” Amendment 01 of the

BAA.

Question 49: What does “interchangeable TA1, TA2, and TA3 approaches” in the BAA mean?

Answer 49: Interchangeable can mean the ability to replace a TA1, TA2 and/or TA3 solution with a better solution during program execution so as to meet the program goals. Please also see

Question 1.

Question 50: Would it still be useful if we provide an abstract on Monday, or would you recommend submitting what we have, albeit less complete, on Friday?

Answer 50: Please see section IV.C.1.a. “Abstract Due Date” of the BAA.