BENGAL QandA Round 1 FINAL 11-29-23 C.pdf

BENGAL

IARPA-BAA-24-03

QUESTIONS & ANSWERS (Round 1)

# Question Answer

1 Must references fit into the 3-page limit? Or is there extra space for references?

White paper requirements are stated in Section 4.1 of the BAA. The page limit for the white papers is 3 pages. References must fit within the three-page limit. Additional pages will not be considered.

2 IARPA often has a requirement that the PI and co-PIS must have 25% effort. I see no statement about this in the BAA. Does this requirement apply for this super seedling?

The BAA provides all the requirements. The Principal Investigator (PI)/Key Personnel role is critical to the success of the overall project and the degree of involvement will be considered when evaluating overall scientific and technical merits. The BAA is hereby amended to clarify. In BAA Section 4.2.1, paragraph A., Proposal Overview, add the following to the project contributor paragraph: “Demonstrate that key personnel/Principal Investigator are sufficiently dedicated to the project.”

In BAA Section 5.1. add the following under paragraph A. Overall Scientific and Technical Merit: "The proposed technical team has the expertise and experience to accomplish the proposed tasks and is sufficiently dedicated to the project”.

3 If we submit the abstract in the early part of the submission window (e.g., Nov 27th), will we get feedback 2 weeks from the submission date (submitted 11/27, feedback provided 12/11) or will all feedback be provided 2 weeks from the final due date for submission (due 12/5, feedback provided 12/19)?

Section 5 of the BAA states: "All efforts will be made to provide a response within 15 days of the due date.” An earlier submission does not mean an earlier response. IARPA's intent is to provide all offerors the same amount of time from white paper notification to proposal due date. The BAA will be clarified.

4 Topic 3 subtopic 1 refers to "unlearning" -- is there a difference between "unlearning" and "correcting" the model?

Yes. The goal of unlearning is to remove inappropriate or sensitive information from the model. Model correction might leave inappropriate or sensitive information in place.

5 The wording also says "IARPA is not interested in filtering outputs". By "filtering outputs" does IARPA mean that it is not interested in approaches for post-generation correction? If so, is this the case for this subtopic or for all relevant subtopics e.g. methods

IARPA is not interested in approaches that focus on filtering outputs (e.g. Regex, keyword filtering, hard-coded filters) in Topic 3.

However, IARPA is interested in other novel approaches for targeted “unlearning” in pre-trained or fine-tuned LLMs that are relevant to this Topic 3 sub-topic.

BENGAL

IARPA-BAA-24-03

QUESTIONS & ANSWERS (Round 1) for reducing hallucinations? Is IARPA interested in pre-generation corrections only?

6 Do Academic Institutions Acknowledgment Letters need to be submitted with White Papers?

If applicable, Academic Institution Acknowledgment Letter(s) will only be required as attachment(s) to the Technical/Management Proposal (BAA Section 4.2.1).

7 Is it desirable to include references of the proposing team's work, and if so, are they excluded from the 3-page count of the White Paper?

See Answer to Question #1.

8 I am unable to access the IDEAS webpage to begin my whitepaper submission. The "Select BAA" field does not list the BENGAL BAA. What should I do?

A posting was issued on SAM.gov on 11/22/23 which explained the delay in IDEAS access. We are working to fix this. It should be ready to accept registration before the submission period for white papers begins on 12/1/23. If there are any further delays, a notification will be issued.

9 Per the solicitation’s instructions it does not appear that this organization will have to include a Transmittal Letter as part of the white paper package, but the IDEAS system requires that one be attached alongside a white paper and completed coversheet form before the whole package can be submitted.

As the solicitation doesn’t mention a Transmittal Letter during the pre-proposal phase (only the full proposal) or provide a template or instructions for one, we are assuming that this a system error. We are happy to put one together or use a placeholding document to satisfy the system’s requirement but wanted to run this by you first.

A transmittal letter is not required for the white paper. The IDEAS system lists the transmittal letter as “optional” because it is required for the proposal (we were limited to this choice by system constraints in the IDEAS set up). The transmittal letter link for the white paper, however, will take the Offeror to a page where they can object to Non- Government personnel seeing their submittal, if that is the case. The white paper is limited to 3 pages. Please follow the BAA instructions for the white paper requirements. Complete the IDEAS coversheet for the white paper (not included in white paper page count). For the proposal, you will also include the technical and cost proposal coversheets that are included in BAA Appendix A.