COOP_QA_5.7.2024.pdf

Continuous-correctness On Opaque Processors (COOP) Frequently Asked Questions (FAQs)

Question #1: Is source or binary legacy compatibility with certain platforms particularly desirable (Operating System (OS), Virtual Machine Monitor (VMM), microkernel, instruction set, etc.)?

Answer #1: Defense Advanced Research Projects Agency (DARPA) cannot comment on specific approaches. DARPA is open to solutions that meet COOP program goals and metrics. As described in the Broad Agency Announcement (BAA), proposers should describe assumptions, risks, and mitigations in the proposal.

Question #2: What types of Controlled Unclassified Information (CUI) data and data formats should we be prepared to process?

Answer #2: There are no specific CUI data formats and data to process. Please refer to Attachment G of the BAA for CUI guidance pertaining to COOP.

Questions #3: Will the program pay for Multi-Project Wafer (MPW) costs or should these be estimated in pricing for Option II?

Answer #3: Please refer to Section 1.4 “Program Structure” of the BAA for information on multi-project wafer services. It is possible that the Government procure MPW services which will allow the Government to exercise Option 2 without exercising Option 3. If the Government is unable to secure or pay for a fabrication and packaging effort, then Option 3 may be exercised so the performer can procure the services themselves.

Question #4: What happens if DARPA picks a MPW process type?

Answer #4: Please consider clarifying the question and resubmitting.

Question #5: In Phase I, can [proposers] use open-source hardware?

Answer #5: Yes, a technical solution may include open-source hardware provided it meets program goals and metrics.

Question #6: Will the slides and proposers list be provided?

Answer #6: The slides will be posted. Please see Section IV of the BAA for attendee list requests.

Question #7: Overhead? Power overhead? Latency overhead? Area overhead? All of the Above?

Answer #7: Please consider clarifying the question and resubmitting.

Question #8: Can a university be a fundamental research sub-contractor to a Prime in this work, or do all publication restrictions have to flow down?

Answer #8: Per Section IV of the BAA, while DARPA does not envision ready opportunities for Fundamental Research, either at the prime contractor or subcontractor level, under COOP, proposers are free to propose Fundamental Research (either by the prime or subcontractor) provided details are included to justify how the inclusion of Fundamental Research will still enable the COOP goals and objectives to be met.

Question #9: In Phase I, are [proposers] going to work with software generated by the Secure Mathematically Assured Composition of Control Models (SMACCM) approach, or is it mentioned so that [proposers] assume the trustworthiness of the control software?

Answer #9: With regards to trustworthiness of the mission critical software, please refer to Section 1.3 “Informal Threat Model” within the BAA. Any assumptions on which the technical approach is based should be described clearly in the proposal. SMACCM is used as an example mission critical software.

Question #10: [Should proposers] assume the presence of a trusted OS/VMM?

Answer #10: Proposals should describe assumptions, the threat model, and control boundaries. This includes which hardware and/or software can be considered opaque.

Question #11: Is the target embedded, general purpose systems or both?

Answer #11: DARPA cannot comment on specific approaches. Please see Section 1.5 “Major Milestones” in the BAA for examples of processor types on which the COOP solutions will be demonstrated.

Question #12: In Phase I – Simulators- Are these built by the performers or Government Furnished Equipment (GFE)?

Answer #12: DARPA cannot comment on specific approaches, to include the purchase/fabrication of devices or requests for Government Furnished Information (GFI)/GFE. Proposals must describe any such needs or assumptions.

Question #13: How do False Positive (FP)/False Negative (FN) targets relate to detection probability target?

Answer #13: As per Section 1.2 “Program Description” of the BAA, the false positive and false negative metrics are assessed using IV&V designed test cases.

Question #14: Can [proposers] rely on new features of upcoming commercial off the shelf processors to achieve COOP program goals?

Answer #14: Proposers can rely on any features of commercial-off-the-shelf processors. However, proposers should describe related risks and mitigations, such as if the features are not in the final product or if the features are disabled.

Question #15: Can changes to processor die be proposed if [proposers] show that they would be suitable for wide commercialization?

Answer # 15: DARPA cannot comment on specific approaches, to include how a proposer may plan to show suitability for wide commercialization. Please describe any risks and mitigations to include the risk of commercialization failure. Also see Question #14.

Question #16: Question about the 99.9999…% accuracy:

is it with respect to a physical model that has been fed to a formal method tool, or it is with respect to the physical world?

In other words, is the gap between the physical world and its formal model included in the 99.99%?

Answer #16: The metric is based on the threat models and assumptions in each proposer’s technical proposal. Please also see the Phase I (Base) paragraph in the “Program Structure” section of the BAA.

Question #17: In one of the “proposed approaches”, namely the “prover-verifier” one, are you referring to “prover” and “verifier” as in

Crypto? As in proof of computation or zero-knowledge (e.g. Securing Information for Encrypted Verification and Evaluation (SIEVE) program @ Information Innovation Office (I20))?

Or as in logic/theory proving? As in proof-scratch and proof-checking (e.g. in a proof assistant)?

Answer #17: DARPA described potential approaches in the slides, not proposed approaches. Please consider rephrasing the question and resubmitting.

Question #18: Can [DARPA] provide example Commercial-off-the-shelf (COTS) Central Processing Units (CPUs) for: single-threaded, multi-threaded, high performance. Out of order? Branch Prediction? CPU Clock Rate?

Answer #18: Proposers should specify which specific processor instances they plan to use. Please also see the “Major Milestones” section in the BAA for examples of processor types on which COOP solutions will be demonstrated.

Question #19: It appears that all common threats that do not involve a computational error are out-of-scope (leakage, software errors, calculated connectivity but leads to underserved operational control floor, ex?) Can DARPA confirm whether this is true?

Answer #19: Please refer to the 2nd paragraph of section 1.3.1 of the BAA.

Question #20: Are [proposers] required to develop a solution that uses a custom Application Specific Integrated Circuit (ASIC) and price it out in option 2&3? Or can [proposers] propose alternate (non-fabrication) approaches?

Answer #20: No, proposers are not required to develop a solution that uses a custom ASIC. Proposers are free to propose any technical solution that will meet program goals and metrics.

Question #21: Are software glitching / fault tolerance approaches in scope?

Answer #21: DARPA cannot comment on specific approaches. Proposers are free to propose any technical solution that will meet program goals and metrics.

Question #22: Are approaches that reduce the ability of the attacker to know if COOP is deployed required? Valuable? Interesting? Out of scope?

Answer #22: DARPA cannot comment on specific approaches. Proposers are free to propose any

Question #23: Can [proposers] focus on processor types possessing certain types of monitoring features, e.g., instruction training?

Answer #23: DARPA cannot comment on specific approaches. Proposers are free to propose any technical solution provided that will meet program goals and metrics.

Question #24: Is a Field Programmable Gate Array (FPGA) with multiple soft cores acceptable for Phase I?

Answer #24: DARPA cannot comment on specific approaches. Proposers are free to propose any

Question #25: Is in-package more desirable than off-package?

Answer #25: DARPA cannot comment on specific approaches. Proposers are free to propose any technical solution provided that will meet program goals and metrics.

Question #26: [Is DARPA] opposed to a chiplet based approach instead of System on Chip (SOC)?

Answer #26: DARPA cannot comment on specific approaches. Proposers are free to propose any

Question # 27: Will [DARPA] accept an open-source process as a component or target of solution?

Answer #27: DARPA cannot comment on specific approaches. Proposers are free to propose any technical solution that will meet program goals and metrics. Please also see Question #5.

Question #28: Is a solution that works for embedded or limited function systems (but not general purpose) acceptable?

Answer #28: DARPA cannot comment on specific approaches. Proposers are free to propose any

Question #29: Will [DARPA] accept a modified processor architecture?

Answer #29: DARPA cannot comment on specific approaches. Proposers are free to propose any technical solution that will meet program goals and metrics. Please also see Questions #14 and #15.

Question #30: FP/FN metrics depend on the base error rate. E.g. if I have 1% error rate, it is trivial to classify with FN=1%. What is the base error rate?

Answer #30: Please see Question #13.

Question #31: Regarding the false negative and false positive rates, what is the countable entity the percentage is calculated against? Percent of what?

Answer #31: Please see Question #13.

Question #32: How does accuracy metric relate to FP/FN metrics?

Answer #32: The accuracy and FP/FN metrics are different. Please refer to the BAA, Questions #13 and #16.

Question #33: [It was] mentioned [during the Proposer’s Day briefing] that this is targeting integrity more so than availability, but what about confidentiality compromises? They seem more difficult to detect over timing/accuracy errors.

Answer #33: Please see Question #19.

Question #34: Please identify the mechanism for proposers to submit abstracts.

Answer #34: Both abstracts and proposals must be submitted using DARPA’s Broad Agency Announcement Tool (BAAT). This can be accessed via the Proposer Instructions: General Terms and Conditions (darpa.mil) page, in the Unclassified Submission Instructions section.

Question #35: Can proposers solely use digital side channels for the COOP solution?

Answer #35: No. As per definitions described in the BAA, using digital-side-channels only implies that the manifestations of software running on hardware are not physical and therefore does not link software with physics. Proposers could use digital-side-channels as part of their proposed solution (i.e., use multi-modal-side-channels).

Question #36: Does COOP have small budget caps (e.g., $5M per performer) like other MVPs (minimal viable programs)?

Answer #36: COOP is a two-phase program as outlined in the “Program Structure” section of the BAA.

Proposers should propose an approach and budget that they believe is necessary to achieve the COOP program goals and metrics rather than attempt to meet a perceived budget cap. Proposed budgets are evaluated in accordance with the “Cost and Schedule Realism” section of “Section II: Evaluation Criteria” in the BAA. A budget that is too high or too low could be evaluated as not realistic.

Question #37: Can [proposers] assume that the adversary is limited to 1800W (120V x 15A) of compute power in my threat model?

Answer #37: Yes, this physical limitation is consistent with the COOP threat model (Section 1.3.2) and COOP boundaries (Section 1.3.3) discussions of the BAA. Proposals that use this or similar definitions in their threat models should also include a discussion on how their technical approach can scale to larger compute environments (e.g., server class power supplies).

Question #38: What does “abstract entity” mean in the “Oracle” definition?

Answer #38: The BAA defines an oracle is “an abstract entity used in formal methods to represent the axiomatic source of correct answers”. “Abstract” means it can be used in formal argumentation but cannot be used in real implementations because real-world oracles do not exist. Proposed solutions that rely on oracles such as "golden processors" to execute mission critical software (MCS) rather than having the opaque processor execute MCS are out of scope for COOP. Approaches that execute a portion of an MCS on an assumed to be trusted processor as part of a formal verification setup are in scope. Proposed solutions such as one that uses an abstract machine implementation of a formal specification as a "golden processor" to reason about important characteristics of the MCS and physics are also within scope.

Question #39: Can [DARPA] provide additional examples of MCS?

Answer #39: In addition to control software such as SMACCM (Secure Mathematically Assured Composition of Control Models), other examples of MCS include, but are not limited to, image processing, PNT (position, navigation, and timing), and policy enforcement software.

Question #40: Is a trusted processor design that is correct-by-construction in scope?

Answer #40: There is insufficient information to answer this question; please consider rephrasing the question and resubmitting if the following response does not answer the intended question. Correct-by-construction solutions are in scope as long as they meet the program goals and metrics. Instances where correct-by-construction solutions are not in scope include, but are not limited to, 1) a trusted processor that serves as a “golden processor” executing MCS rather than having the opaque processor execute MCS (see Question #38); 2) correct-by-construction techniques does not link MCS to physics; 3) a trusted processor design that is limited to a single type of digital processor.

Question #41: I already accomplished the program goals as part of [another research effort], is there a way to bid to directly commercialize the technology?

Answer #41: No, all proposals must be submitted and evaluated in accordance with the BAA. Proposers should not assume that reviewers are familiar with the team's previous work or results. Proposers should not assume that reviewers will read any or all references. The proposal narrative should include a description of what the referenced technique, insight, or result is and why it applies to COOP rather than stating the claim as fact.

Question #42: Can [DARPA] clarify what a proposed threat model should consist of?

Answer #42: The threat model is the foundation for the logical presentation of the proposed technical approach. It should contain 1) a clear description of what properties are needed to satisfy the program goals; 2) a clear description of fundamental assumptions including a description of the oracle(s) being used (if applicable) and a list of trusted and untrusted components (this helps set the boundaries to which the proposed approach is valid); 3) a clear description of the threats and their capabilities (this further bounds the problem by setting limitation on the what adversary can do). The rest of the proposal should build upon this threat model.

Question #43: Can [proposers] assume that the source code for MCS is available?

Answer #43: Yes. If proposers make this assumption, it should be clearly stated as part of the threat model.

Question #44: Can [proposers] assume that the operating system is cooperative? For example, interrupts will be processed correctly and on time?

Answer #44: As defined in the BAA, non-MCS including the operating system are opaque and therefore deterministic. Deterministic does not mean correct and on time. In the example, interrupts may be processed incorrectly or at the wrong time, but this erroneous behavior is deterministic. Similarly, the erroneous behavior might be the result of a bug or an attack by an adversary. In either case, the behavior and result are still deterministic. In this sense, the operating system is neither cooperative (as in helpful) nor non-cooperative.

Question #45: Can [proposers] use a published threat model instead of presenting a new one?

Answer #45: Yes, proposers can use a published threat model as long as it is also clearly described in the required threat model section. Proposers should not assume that reviewers will read any or all references. Please also see Question #41.

Question #46: The SOW guidelines indicate that we should identify if unclassified tasks are proposed as fundamental research or CUI. We have some tasks that are not fundamental research but are not necessarily being proposed as CUI. How should we identify those tasks? Should we indicate 'Not Fundamental Research' for those tasks?

Answer #46: Only fundamental research and CUI need to be identified. Portions of the SOW that do not fit either do not need to be identified.

Question #47: Are we required to submit the 2024 reps and certs or the 2020 reps and certs? Is it required that the prime applicant and all subcontractors complete the reps and certs file or just the prime applicant?

Answer #47: Only the prime is required to submit the 2024 Procurement Contracts Reps and Certs document. This document can be found in the “Procurement Contracts Representation and Certifications” section of the darpa.mil page (https://www.darpa.mil/work-with-us/procurement-contracts.) The 2020 Procurement Reps and Certs document is not required.

Question #48: Can you advise on how we should handle submission if our cost spreadsheet file pushes us over the submission size limit? Should we email our subcontractor cost proposals separately?

Answer #48: Proposers (including subcontractors) must submit all parts of their submission using the same method, being mindful of the file size limit when utilizing BAAT. No e-mail submissions will be accepted with the exception of subcontractor submissions of the DARPA Standard Cost Proposal Spreadsheet. Please be mindful of the proposal deadline of May 13 at 4 pm ET when submitting your proposal through BAAT.

Question #49: Are proposers required to include additional Options 2 and 3 in their proposals?

Answer #49: Proposers are not required to include pricing for Option 2 and Option 3. The COOP program seeks to demonstrate effectiveness on real multi-threaded core hardware in Phase II and proposers can include all fabrication and packaging costs in their base and Option 1 proposals. Priced Option 2 and Option 3 helps DARPA make informed decisions on whether and how to provide Government-funded fabrication and packaging activities. Given that program goals and metrics can be met, a proposal may be selected for negotiations even if Option 2 and Option 3 are not priced.

Question #50: Are proposed approaches for Option 2 required to exclusively use GDSII files?

Answer #50: No. Any relevant design files or formats may be utilized as part of the proposed approach.

Question #51: Can proposals for Option 2 (Detailed design) and Option 3 (Fabrication) include costs for non-chip based approaches (e.g., board-based approaches or hybrid board and chip-based approaches)?

Answer #51: Board-based approaches and any non-chip fabrication and packaging related costs in hybrid board and chip-based approaches should include all associated costs in the Base and Option 1 proposals. These costs should not be included in the Option 2 and Option 3 proposals.