SoURCE CODE BAA QnA ROUND 1 - 12132023_C.docx

SOURCE SELECTION INFORMATION - SEE FAR 2.101 AND 3.104.

SoURCE CODE Q&As ROUND 1

1. The topic is the citizenship restriction on TA1-T2-TA3 bids. Example: University PI is prime and company X is sub on TA1 bid, and Company X is prime and same university-PI is sub on TA1, TA2, TA3 bid. Will the government accept proposals for both these bids, not just one? Will the government allow awards to both these bids, not just one? If all are not acceptable to gov't, it restricts the nature of partnerships and collaborations across industry and universities. Is this restriction intentional?

Answer 1. Yes, the Government would accept proposals for both these bids provided there were no other restrictions (i.e. duplicative funding, eligibility of applicants, conflicts of interest., etc.) See BAA Sections 3.A.2. Multiple Submissions to the BAA and 3.C.1 Collaboration Efforts for guidance. There are also restrictions related to the classified portions of the Program as set forth in BAA Section 3.A, Eligible Applicants, see excerpt below, and 4.A, Approval to Receive Classified Addendum and Prepare and Submit a Classified Proposal.

From BAA Section 3.A:

For vendors who are ONLY proposing to the unclassified TA1 portion of this effort: Foreign entities and/or individuals may participate, but only as a part of a U.S. based team. The prime contractor must be a U.S. organization. Foreign entities and individuals may participate as subcontractors or employees of a U.S. organization; however, all foreign participation must comply with any necessary Non-Disclosure Agreements, Security Regulations, Export Control Laws, and other governing statutes applicable under the circumstances. Offerors are expected to ensure that the efforts of foreign participants do not either directly or indirectly compromise the laws of the United States, nor its security interests. As such, both foreign and domestic Offerors should carefully consider the roles and responsibilities of foreign participants as they pursue teaming arrangements.

The BAA will be updated to read the following: For vendors who are proposing to all Technical Areas – TA1 and the classified Areas, TA 2 and TA3, of this effort: Entities under unmitigated Foreign Ownership, Control, or Influence (FOCI) and/or foreign individuals are NOT eligible to participate in the CLASSIFIED aspects of program, in any capacity. This restriction extends to both classified technical and administrative support roles for the classified technical areas. This restriction does not extend to unclassified technical and administrative support roles for the unclassified technical area. This restriction also extends to foreign national US Person green card holders employed by U.S.-based organizations. All personnel supporting the classified aspects of the program, directly or indirectly, must be a US Citizen. These restrictions extend to subcontractors and/or lower-tier subcontractors for the classified aspects of the program. This does NOT preclude the involvement of foreign nationals in the unclassified aspects of the program (TA1), but vendors security plans should explain how they are going to ensure classified aspects of the program are properly protected from foreign nationals on TA1.

2. Company X is interested in the classified BAA for this effort. Do we need to resubmit the request using the guidelines outlined in section 4A or the unclassified BAA?

Answer 2: Yes, any company that wants to receive the Classified Addendum to the BAA must submit a request in accordance with the requirement of Section 4.A, Approval to Receive Classified Addendum and Prepare and Submit a Classified Proposal, IARPA is extending the due date for receipt of these requests. See BAA Amendment 1.

3. For vendors who are proposing to all Technical Areas – TA1 and the classified Areas, TA 2 and TA3, of this effort: Entities under unmitigated Foreign Ownership, Control, or Influence (FOCI) and/or foreign individuals are NOT eligible to participate in the program in any capacity. This restriction extends to both unclassified technical and administrative support roles. This restriction also extends to foreign national US Person green card holders employed by U.S.-based organizations. All personnel supporting the program directly or indirectly must be a US Citizen. These restrictions extend to subcontractors and/or lower-tier subcontractors. Question: Should we interpret the above paragraph to mean that a U.S.-based prime contractor leading a proposal that covers all Technical Areas of SoURCE CODE cannot have a university subcontractor perform on the TA1 portion of their proposal if the university will fund foreign students on the program?

Answer 3: No. Entities under unmitigated Foreign Ownership, Control, or Influence (FOCI) and/or foreign individuals are NOT eligible to participate in the program in classified aspects of the program. Please see answer to number 1.

4. Would the government consider adjusting the eligible applicant's criteria for all-TA bids to match that of TA1-only bids where FOCI and/or foreign individuals may participate but only on TA1 to ensure fair competition between TA1-only and all-TA bids as well as promote academic involvement?

Answer 4: Please see answer to number 1.

5. The BAA specifies that potential vendors may propose to either (1) TA1-only or (2) all Technical Areas [BAA, Section 2, p. 17]. It goes on to say, that for vendors proposing to all Technical Areas, all personnel supporting the program directly or indirectly must be a US Citizen (no Green Card holders), and that these restrictions extend to subcontractors and/or lower-tier subcontractors [BAA, Section 3A, p. 18]. This restriction effectively blocks teams who plan to propose all technical areas from including academic institutions, even if the academic institutions are relegated to TA1 tasks and firewalled from TAs 2 and 3. Please confirm this is the intent of the BAA.

Answer 5: Please see answer to number 1.

6. Before we submit a formal request for the classified addendum (TA2 and TA3), we have a few questions for IARPA. 1. What is the due date for the TA2 and TA3 submissions? Is it also January 22, 2024, or a different deadline? 2. Is the classified addendum SECRET//NOFORN or a higher classification level? 3. Can the classified addendum be accessed outside of NSA ARC (e.g., courier, in-person)? We are having issues accessing the classified NSA ARC, but can courier and properly store classified documents at our facility and are also nearby to visit IARPA in-person to review the addendum, if necessary. We appreciate the clarification and look forward to hearing from you.

Answer 6: The BAA will be amended to extend the proposal due date for TAI/TA2/TA3 submissions. The Classified Addendum is SECRET//NOFORN and will not be released on the NSA ARC. See Section 4.A for instructions on requests to Receive the Classified Addendum and Prepare and Submit classified proposals. Once approval to receive and submit classified information has been determined, more information will be provided regarding options for obtaining, storing and submitting the classified proposals. Note: The BAA will be amended to extend the due date for receipt of these requests. See BAA Amendment 001.

7. The text in this section notes, The T&E Team will conduct dataset development activities throughout the life of the program using Institutional Review Board (IRB)-approved Human Subject Research (HSR) protocols. A portion of the data will be held out for T&E, the remaining portion will be made available to Performers for R&D. Can the government clarify if the data or datasets that will be provided to the performer teams will include human subject data?

Answer 7: Details about the datasets will be provided at program kick-off.

8. Will the data supplied by the T&E team be labeled, and if so, what labeling schema will be used?

Answer 8: See number 7

9. How detailed will the annotations provided by the T&E team be in terms of demographic information?

Answer 9: See number 7

10. Will the government provide greater clarity (or provide the reference architecture) as to the baseline models or reference systems that will be provided against which our proposed solutions can be compared during the evaluation phase?

Answer 10: We will provide information on baselines at kickoff.

11. Are there any limitations or specific guidelines on the use of open-source software or libraries in the development of the proposed system?

Answer 11: No

12. Recognizing the importance of data augmentation in classification tasks, would the government confirm that binary diversification techniques are within scope?

Answer 13: Datasets will be created by T&E teams.

13. Leveraging existing and highly functional commercial systems and / or software can help speed performance and lower cost. Applying and integrating these with newly innovative capabilities will further enhance performance. Can the government clarify that integrating commercially available systems and software with newly created platforms, systems, and software will be considered in scope?

Answer 14: The Government is looking for innovative approaches to meet program goals and milestones.

14. With the understanding that the Cover Page and Transmittal Letters cannot exceed 1 page each, can the government clarify if they count against the 25-page maximum page limit for TA1?

Answer 14: They do not count against the page limit.

15. 1) For universities that only conduct “fundamental research” as defined in U.S. export control regulations and NSDD 189, will IARPA fund projects scoped as “fundamental research” under TA 1? 2) For projects under TA 1, will there be a process whereby IARPA will approve foreign national researchers prior to their participation on the project?

Answer 15: Performers will be responsible for complying with export control regulations (i.e., ITAR and EAR). See BAA Section 6.B.6 Export Control. Regarding TA1 and foreign researchers, see the BAA sections 3.A Eligible Applicants and 4.A Approval to Receive Classified Addendum and Prepare and Submit a Classified Proposal and the answer to number 1 above.

16. I want to ask for clarification about the expected budget materials for our submission in January. Do we need to submit concrete numbers with justifications or rough numbers with much simpler templates?

Answer 16: The BAA initially requests cost, schedule and milestones and technical resource information as part of the Volume I technical proposal. Full cost proposals with all supporting information (Volume II) will be requested by the Contracting Officer if your technical proposal is selected for award. See BAA Sections 4.B.1. Volume I: Technical and Management Proposal, 4.B.2 Volume II: Cost Proposal and 5.C. Negotiation and Contract Award. Note: If proposed costs in Volume II are substantially different than the estimates provided in the technical proposal, then a contract may not be awarded.

17. What is the scale of the provided dataset? Is the dataset labeled with author/demographic/other information? How much of the dataset is source vs binary? Are there any source code and binary that represent the same program? Is this labeled in the dataset? Are there any build tools or readmes provided with source code? Are these entire repositories or single files? Are there any differences between the provided and testing datasets? Particularly concerning false positives. Will the dataset contain adversary-generated false positives? For example, a malware author can purposefully write code to appear as a false target or utilize obfuscation techniques - is this type of binary/code in scope? Is the dataset curated for a particular technical target, e.g., Windows or Linux systems? If the dataset is labeled with demographic information, is the dataset curated for a particular target, e.g., ransomware gangs.

Answer 17: See number 7

18. From Figure 5 (excerpt above) in the BAA section 1.E, TA1 similarity measures are run on unseen data to “induce demographics” by being provided “Rank List similar binaries”; are the “induced demographics” defined by IARPA or by the performer? For programs with multiple authors, is the goal to identify demographic profiles for individual authors or for the group as a whole?

Answer 18: 1) The induced demographics will be defined based on the annotations in the datasets. 2) More details on multiple author sets will be provided through the program - minimally, group details, but program may explore ability to list individual authors within the group.

19. Can we include non-binary/non-source code data from external sources, if assessment of binary/source code data informs assessment of the non-binary/non-source code data?

Answer 19: Please follow the directions in the BAA section 1.D.1.b Performer Supplemented Data Sources.

20. The BAA states TA1 must provide tools for featurization of both source & binaries; are there any restrictions on the structure of the features? Will there be a defined data structure for features or for any other elements of the T&E framework?

Answer 20: 1) No restrictions; We are looking for innovative solutions. 2) More details about the T&E framework (esp. APIs) will be provided at kickoff. You can assume standard programming practices where Offerors will be evaluated based on performer containerized applications.

21. On page 18, you specify “For vendors who are proposing to all Technical Areas – TA1 and the classified Areas, TA 2 and TA3, of this effort: Entities under unmitigated Foreign Ownership, Control, or Influence (FOCI) and/or foreign individuals are NOT eligible to participate in the program in any capacity. This restriction extends to both unclassified technical and administrative support roles. This restriction also extends to foreign national US Person green card holders employed by U.S.-based organizations. All personnel supporting the program directly or indirectly must be a US Citizen. These restrictions extend to subcontractors and/or lower-tier subcontractors.” a. Does this apply if the Prime is planning to subcontract to a US owned University subcontractor who may have foreign students but is firewalled for support exclusive to TA1, and the prime meets all criteria for TA2 and TA3? Can security concerns be mitigated by a program protection plan and OPSEC plan?

Answer 21: See answers to number 1.

22. 1. Is PDF an acceptable format for submission for all files (aside from the estimated cost breakdown to be provided in excel)? 2. Are there any file size limits for your email system?

Answer 22: 1) Yes 2) Approximately 20MB

23. 1. In figure 5, upper right, it states ‘Test feature invariance across datasets.’ Can you please explicitly define what you mean by ‘feature invariance’? 2. In figure 5, middle, an image is labelled 'Induce Demographics'. Will the performer be able to supply a separate model that produces demographic outputs, or will T&E induce the demographics from the similarity vectors?

Answer 23: 1) We will be testing it across several datasets. We are not trying to optimize against specific datasets but would like any identified features to work across any number of datasets. 2) A separate model for demographics may be allowed. We are looking for innovative solutions.

24. 1) 4. B.2.c. Section 3: Supporting Information – Can supporting information be provided in PDF format, except for Subcontractors cost breakdowns, which will be provided in the requested Excel format? 2) Can the input format and usage of the SoURCE software be elaborated on? For example, will we be provided a set of binaries/documents in one call {A, B, C} that we are expected to return results on, and can we expect reuse of those binaries/documents in another call {B, C, D}? Is the SoURCE software intended to be used over any number of queries or is it meant to be called a single time and terminated?

Answer 24: 1) Yes, except for cost breakdowns that are required to be in Excel. 2) More details will be provided at kick-off, especially on the API. However, you can expect that we will minimally provide one source/binary and expect the performer system to output the most similar, similarity score (or some other method for measuring how they are closely related) and some explanation/evidence to support the claim from the system. Performers will containerize and we will make multiple calls to the container for test and evaluation purposes, which will require your system to be able to handle the automated test suite - details provided at kickoff. Outside of this, how a performer system handles this is entirely up to you as the offeror. We are looking for innovative solutions.

25. 1. Should offerors who plan to submit against TAs 1, 2 and 3 submit a standalone TA 1 unclassified response while awaiting access to, and direction on, TA 2 and 3 submittal? 2. Are offerors who submit a TA1 proposal, and later decided to submit a TA2 and TA3 proposal based on reading the classified addendum, eligible to do so?

a. 1. See Section 3.A.2 Multiple Submissions to the BAA.

b. 2. See Section 3.A.2 Multiple Submissions to the BAA.

26. Is the Government able to provide any estimation on the quantity of data (Source code and binaries) that will be provided in TA1?

Answer 26: See number 7.

27. Do we anticipate the datasets provided to TA1 providers will include any information outside of code (binary/source) and demographic labels? If so, what additional details can be provided? Are there any example data samples that could be provided?

Answer 27: See number 7.

28. 1. Can we assume that the source/binary programs are well-formed? That is, can code from compliable languages be compiled, and interpreted or compiled code execute on some system. 2. What information about the environments or recipes for compiling and executing code will be provided to performers?

Answer 28: 1) We expect data to be well-formed, however, aspects of the source code datasets may contain code samples as well. (We would note any that we know cannot be compiled). 2) See number 7.

29. Is the development and curation of new datasets (potentially making use of HSR) in-scope for TA1 performers? Please note that we are not planning to propose the purchase of data as an ODC.

Answer 29: This is out of scope and is a T&E task.

30. At what point in the program can performers expect to receive initial experimental datasets?

Answer 30: An initial dataset would be provided at the program kick-off.

31. What are the anticipated limits on scope, completeness and size of the source/binaries that should be analyzed? Should performers expect to analyze ‘complete’ programs, code snippets or both? How large (approx. lines of source or executable code) do we expect artifacts to be? What sort, and how many, external dependencies do we expect artifacts to exhibit (ex. linked libraries, software dependencies)? How do these aspects interact with specified program metrics (e.g. processing time)?

Answer 31: See number 7.

32. Will performers be provided with information about the build or execution environment associated with an artifact?

Answer 32: See number 7.

33. The BAA states that TA 1 only proposals are due January 22, 2024. However, since we plan to propose to all TAs, we intend to submit a single classified proposal by the due date in the forthcoming classified addendum to the BAA. Are we correct in assuming that nothing is due from a combined TA1/TA2/TA3 proposal on January 22nd?

Answer 33: Yes, the BAA will be amended to clarify.

34. Figure 5 explicitly separates "feature space containers" from "similarity algorithm containers". Is that a requirement or is it simply illustrative? In Figure 5, are containers supposed to be interchangeable components, potentially allowing T&E to use any similarity container with any feature container? Figure 5 puts "induce demographics" into the "similarity algorithms" container. Is that a requirement or simply illustrative? Can "induce demographics" use more than "rank list of similar binaries" as input? In Figure 5, supposing the suggested containers are interchangeable components, would "induce demographics" potentially merit its own containers? Please confirm receipt of this email.

Answer 34: Figure 5 is illustrative. We are looking for innovative solutions.

35. By when do you expect to send the classified addendum to qualifier offerors? Will you allow offerors to submit separate TA1 and TA2/TA3 proposals? Stated differently, are offerors permitted to submit a proposal only covering TA2/TA3?

Answer 35: The Classified Addendum will be sent to qualified offerors once Security has completed its review of submitted documentation, See BAA Section 4.A Approval to Receive Classified Addendum and Prepare and Submit a Classified Proposal. The BAA will be amended to clarify the proposal due dates.

36. Will IARPA provide a template for Data Privacy Plan v1.0 referenced in Section 1.D.1.b of the BAA?

Answer 36: No.

37. Is there any way that the classified documents (e.g., submitted proposal volumes) can be transferred electronically via JWICS or other system?

Answer 37: The Government will provide specific instructions to Offerors, who submitted requests, and are approved to submit classified proposal documents.

38. Will a classified combined technical volume responding to TA1, TA2, and TA3 be due on 22 JAN 2024 (per the BAA, for TA1 only submissions) or a later date as the release of classified addendum takes place after the release of the unclassified BAA?

Answer 38: The BAA will be amended to extend the due date. See answer to question 2.

39. In the event of selection for negotiation and the final cost volume is submitted, will IARPA allow the final cost in the cost volume to deviate, within reason or a certain percentage, from the initial cost submitted (Total Funds Requested on the cover page) with the proposal?

Answer 39: Yes. If proposed costs submitted are substantially different than the estimates provided in the technical proposal, then a contract may not be awarded. See BAA Section 5.C Negotiation and Contract Award.

40. Company X has not yet finalized its subcontractors for the classified TAs. Will we need to submit a subcontractor’s SF328, the FOCI form, to IARPA after they join our team?

Answer 40: Yes.

SOURCE SELECTION INFORMATION - SEE FAR 2.101 AND 3.104.