Bias Effects and Notable Generative AI Limitations (BENGAL) Targeted Super Seedling Research Topics
Closed Solicitation Posted
- Solicitation number
- IARPA-BAA-24-03
- Agency
- Intelligence Advanced Research Projects Activity Office of the Director of National Intelligence
- Responses due
- Set-aside
- No set-aside
Opportunity facts
- NAICS code
- 541715 Research and Development in the Physical, Engineering, and Life Sciences (except Nanotechnology and Biotechnology)
- PSC
- AJ12 General Science And Technology R&D Services; General Science And Technology; Applied Research
- Place of performance
- Washington, District of Columbia 20511, United States
Notice details come from SAM.gov. Updated .
About this opportunity
The Intelligence Advanced Research Projects Activity seeks proposals to conduct early-stage research projects focused on understanding and addressing bias, hallucinations, safe information flow, and limitations in large language models through its Bias Effects and Notable Generative AI Limitations Targeted Super Seedling solicitation. Offerors are encouraged to first submit white papers by December 5, 2023 outlining their qualifications and technical approach, with proposals due February 9, 2024. Awards will be made as procurement contracts valued at up to $4 million per proposal for research conducted over 12 to 24 month periods, with evaluation considering overall scientific and technical merit and relevance to the agency's mission.
There is no set-aside designation. Research should be conducted within the 541712 NAICS code and focus on one or more of the R423 PSC code topic domains, including biases and diverse perspectives, AI hallucinations and inferences, safe information flow, and working with imperfect sources. Place of performance will be in Washington, D.C. and include limited travel for kickoff meetings and site visits. Multiple awards are anticipated to further the agency's goal of developing new generative AI capabilities.
Notice text
8 versions
Update #8 · Latest ·
1/24/2024 – SAM.gov due date has been updated to 2/9/2024.
1/24/2024 – Questions and Answers (Q&A) Round #5 Posting and Amendment 005:
BENGAL BAA Q&A (Round #5) answers to Questions 41 & 42 are attached. The BAA is amended to: 1) extend the due date for the proposals to February 9, 2024 4:00 PM EST and 2) incorporate the answer/change to Question #41 from the BENGAL Questions & Answers (Round 5). A full, updated BAA is attached.
12/20/2023 – Questions and Answers (Q&A) Round #4 Posting and Amendment 004:
BENGAL BAA Q&A (Round #4) answers to Questions 15 – 40 are attached. The BAA is amended to: 1) incorporate the answers/changes for Question #21 from the BENGAL Q&A (Round #4) to BAA Section 4.2.1, Volume 1 – Technical & Management Proposal; 2) incorporate the answers/changes for Question #24 from the BENGAL Q&A (Round #4) to BAA Section 4.2.2, Cost/Price Proposal, Paragraph B; 3) correct BAA # on Appendix A. 1; and 4) provide Word document templates for the BENGAL BAA appendices. A full, updated BAA is attached.
NOTE: Any remaining BENGAL BAA questions will be answered near-term. Offerors will receive White Paper responses NLT 30 days prior to the proposal due date.
12/4/23 - Round 3 QandAs are attached
11/30/23 - Round 2 QandAs are attached
11/29/23 - Amendment 003: Round 1 Question and Answers (QandAs) are attached. The BAA is amended to: 1) extend the due date for the proposals to January 29, 2024 4:00 PM EST; 2) formally incorporate the changes from the BENGAL QandAs (Round 1); 3) update Section 4, Application and Submission Information, list of the Non-Government entities that will have access to proposals, and 4) correct file name for Amendment No. 002 to read “IARPA-BAA-24-03” and issuance dates to reflect “2023” for Amendment Nos. 001 and 002. A full, updated BAA is attached.
NOTE: The IARPA Distribution and Evaluation System (IDEAS) opened on November 29, 2023 for registration and subsequent submission of white papers starting on December 1, 2023. See Question #9 for clarification regarding the transmittal letter instructions for the White Papers and Proposals.
11/22/23 - Amendment 002: The BAA is amended for clarification, to revise the technical section, and to incorporate changes from Amendment 001. A full, updated BAA is attached. NOTE: The IDEAS system is not yet open for submission of white papers. It is anticipated that this will occur early next week. If there are any further delays we will issue a notice here.
11/20/23 - Amendment 001: The BAA is amended to revise the submission window for white papers to December 1, 2023 12:00PM EST through December 5, 2023 12:00PM EST and to add the requirement for a Quad Chart attachment to the proposal, see attached Amendment 001.
________________________________________
Original posting:
White Papers (abstracts) Due Date: December 5, 2023 12:00 PM EST (Offerors are strongly encouraged to submit white papers before submitting a proposal). The submission window for white papers will open on November 27, 2023 12:00PM EST and close on December 5, 2023 12:00PM EST.
The Intelligence Advanced Research Projects Activity (IARPA) invests in high-risk/highpayoff research programs that have the potential to provide our nation with an overwhelming intelligence advantage. IARPA seeks to develop new capabilities to enable the safe adoption and use of generative AI technologies to greatly enhance the effectiveness and efficiency of the Intelligence Community (IC).
The goal of the BENGAL targeted super seedling is to understand LLM threat modes, quantify them and to find novel methods to correct threats and vulnerabilities or to work resiliently with imperfect models. IARPA seeks to develop and incorporate novel technologies to efficiently probe large language models to detect and characterize LLM threat modes and vulnerabilities. Performers will focus on one or more of the topic domains below, clearly articulate a taxonomy of threat modes within their domain of interest and develop technologies to serve as an analog to ‘virus scan’ software.
Update #7 ·
12/20/2023 – Questions and Answers (Q&A) Round #4 Posting and Amendment 004:
BENGAL BAA Q&A (Round #4) answers to Questions 15 – 40 are attached. The BAA is amended to: 1) incorporate the answers/changes for Question #21 from the BENGAL Q&A (Round #4) to BAA Section 4.2.1, Volume 1 – Technical & Management Proposal; 2) incorporate the answers/changes for Question #24 from the BENGAL Q&A (Round #4) to BAA Section 4.2.2, Cost/Price Proposal, Paragraph B; 3) correct BAA # on Appendix A. 1; and 4) provide Word document templates for the BENGAL BAA appendices. A full, updated BAA is attached.
NOTE: Any remaining BENGAL BAA questions will be answered near-term. Offerors will receive White Paper responses NLT 30 days prior to the proposal due date.
12/4/23 - Round 3 QandAs are attached
11/30/23 - Round 2 QandAs are attached
11/29/23 - Amendment 003: Round 1 Question and Answers (QandAs) are attached. The BAA is amended to: 1) extend the due date for the proposals to January 29, 2024 4:00 PM EST; 2) formally incorporate the changes from the BENGAL QandAs (Round 1); 3) update Section 4, Application and Submission Information, list of the Non-Government entities that will have access to proposals, and 4) correct file name for Amendment No. 002 to read “IARPA-BAA-24-03” and issuance dates to reflect “2023” for Amendment Nos. 001 and 002. A full, updated BAA is attached.
NOTE: The IARPA Distribution and Evaluation System (IDEAS) opened on November 29, 2023 for registration and subsequent submission of white papers starting on December 1, 2023. See Question #9 for clarification regarding the transmittal letter instructions for the White Papers and Proposals.
11/22/23 - Amendment 002: The BAA is amended for clarification, to revise the technical section, and to incorporate changes from Amendment 001. A full, updated BAA is attached. NOTE: The IDEAS system is not yet open for submission of white papers. It is anticipated that this will occur early next week. If there are any further delays we will issue a notice here.
11/20/23 - Amendment 001: The BAA is amended to revise the submission window for white papers to December 1, 2023 12:00PM EST through December 5, 2023 12:00PM EST and to add the requirement for a Quad Chart attachment to the proposal, see attached Amendment 001.
________________________________________
Original posting:
White Papers (abstracts) Due Date: December 5, 2023 12:00 PM EST (Offerors are strongly encouraged to submit white papers before submitting a proposal). The submission window for white papers will open on November 27, 2023 12:00PM EST and close on December 5, 2023 12:00PM EST.
The Intelligence Advanced Research Projects Activity (IARPA) invests in high-risk/highpayoff research programs that have the potential to provide our nation with an overwhelming intelligence advantage. IARPA seeks to develop new capabilities to enable the safe adoption and use of generative AI technologies to greatly enhance the effectiveness and efficiency of the Intelligence Community (IC).
The goal of the BENGAL targeted super seedling is to understand LLM threat modes, quantify them and to find novel methods to correct threats and vulnerabilities or to work resiliently with imperfect models. IARPA seeks to develop and incorporate novel technologies to efficiently probe large language models to detect and characterize LLM threat modes and vulnerabilities. Performers will focus on one or more of the topic domains below, clearly articulate a taxonomy of threat modes within their domain of interest and develop technologies to serve as an analog to ‘virus scan’ software.
Update #6 ·
12/4/23 - Round 3 QandAs are attached
11/30/23 - Round 2 QandAs are attached
11/29/23 - Amendment 003: Round 1 Question and Answers (QandAs) are attached. The BAA is amended to: 1) extend the due date for the proposals to January 29, 2024 4:00 PM EST; 2) formally incorporate the changes from the BENGAL QandAs (Round 1); 3) update Section 4, Application and Submission Information, list of the Non-Government entities that will have access to proposals, and 4) correct file name for Amendment No. 002 to read “IARPA-BAA-24-03” and issuance dates to reflect “2023” for Amendment Nos. 001 and 002. A full, updated BAA is attached.
NOTE: The IARPA Distribution and Evaluation System (IDEAS) opened on November 29, 2023 for registration and subsequent submission of white papers starting on December 1, 2023. See Question #9 for clarification regarding the transmittal letter instructions for the White Papers and Proposals.
11/22/23 - Amendment 002: The BAA is amended for clarification, to revise the technical section, and to incorporate changes from Amendment 001. A full, updated BAA is attached. NOTE: The IDEAS system is not yet open for submission of white papers. It is anticipated that this will occur early next week. If there are any further delays we will issue a notice here.
11/20/23 - Amendment 001: The BAA is amended to revise the submission window for white papers to December 1, 2023 12:00PM EST through December 5, 2023 12:00PM EST and to add the requirement for a Quad Chart attachment to the proposal, see attached Amendment 001.
________________________________________
Original posting:
White Papers (abstracts) Due Date: December 5, 2023 12:00 PM EST (Offerors are strongly encouraged to submit white papers before submitting a proposal). The submission window for white papers will open on November 27, 2023 12:00PM EST and close on December 5, 2023 12:00PM EST.
The Intelligence Advanced Research Projects Activity (IARPA) invests in high-risk/highpayoff research programs that have the potential to provide our nation with an overwhelming intelligence advantage. IARPA seeks to develop new capabilities to enable the safe adoption and use of generative AI technologies to greatly enhance the effectiveness and efficiency of the Intelligence Community (IC).
The goal of the BENGAL targeted super seedling is to understand LLM threat modes, quantify them and to find novel methods to correct threats and vulnerabilities or to work resiliently with imperfect models. IARPA seeks to develop and incorporate novel technologies to efficiently probe large language models to detect and characterize LLM threat modes and vulnerabilities. Performers will focus on one or more of the topic domains below, clearly articulate a taxonomy of threat modes within their domain of interest and develop technologies to serve as an analog to ‘virus scan’ software.
Update #5 ·
11/30/23 - Round 2 QandAs are attached
11/29/23 - Amendment 003: Round 1 Question and Answers (QandAs) are attached. The BAA is amended to: 1) extend the due date for the proposals to January 29, 2024 4:00 PM EST; 2) formally incorporate the changes from the BENGAL QandAs (Round 1); 3) update Section 4, Application and Submission Information, list of the Non-Government entities that will have access to proposals, and 4) correct file name for Amendment No. 002 to read “IARPA-BAA-24-03” and issuance dates to reflect “2023” for Amendment Nos. 001 and 002. A full, updated BAA is attached.
NOTE: The IARPA Distribution and Evaluation System (IDEAS) opened on November 29, 2023 for registration and subsequent submission of white papers starting on December 1, 2023. See Question #9 for clarification regarding the transmittal letter instructions for the White Papers and Proposals.
11/22/23 - Amendment 002: The BAA is amended for clarification, to revise the technical section, and to incorporate changes from Amendment 001. A full, updated BAA is attached. NOTE: The IDEAS system is not yet open for submission of white papers. It is anticipated that this will occur early next week. If there are any further delays we will issue a notice here.
11/20/23 - Amendment 001: The BAA is amended to revise the submission window for white papers to December 1, 2023 12:00PM EST through December 5, 2023 12:00PM EST and to add the requirement for a Quad Chart attachment to the proposal, see attached Amendment 001.
________________________________________
Original posting:
White Papers (abstracts) Due Date: December 5, 2023 12:00 PM EST (Offerors are strongly encouraged to submit white papers before submitting a proposal). The submission window for white papers will open on November 27, 2023 12:00PM EST and close on December 5, 2023 12:00PM EST.
The Intelligence Advanced Research Projects Activity (IARPA) invests in high-risk/highpayoff research programs that have the potential to provide our nation with an overwhelming intelligence advantage. IARPA seeks to develop new capabilities to enable the safe adoption and use of generative AI technologies to greatly enhance the effectiveness and efficiency of the Intelligence Community (IC).
The goal of the BENGAL targeted super seedling is to understand LLM threat modes, quantify them and to find novel methods to correct threats and vulnerabilities or to work resiliently with imperfect models. IARPA seeks to develop and incorporate novel technologies to efficiently probe large language models to detect and characterize LLM threat modes and vulnerabilities. Performers will focus on one or more of the topic domains below, clearly articulate a taxonomy of threat modes within their domain of interest and develop technologies to serve as an analog to ‘virus scan’ software.
Update #4 ·
11/29/23 - Amendment 003: Round 1 Question and Answers (QandAs) are attached. The BAA is amended to: 1) extend the due date for the proposals to January 29, 2024 4:00 PM EST; 2) formally incorporate the changes from the BENGAL QandAs (Round 1); 3) update Section 4, Application and Submission Information, list of the Non-Government entities that will have access to proposals, and 4) correct file name for Amendment No. 002 to read “IARPA-BAA-24-03” and issuance dates to reflect “2023” for Amendment Nos. 001 and 002. A full, updated BAA is attached.
NOTE: The IARPA Distribution and Evaluation System (IDEAS) opened on November 29, 2023 for registration and subsequent submission of white papers starting on December 1, 2023. See Question #9 for clarification regarding the transmittal letter instructions for the White Papers and Proposals.
11/22/23 - Amendment 002: The BAA is amended for clarification, to revise the technical section, and to incorporate changes from Amendment 001. A full, updated BAA is attached. NOTE: The IDEAS system is not yet open for submission of white papers. It is anticipated that this will occur early next week. If there are any further delays we will issue a notice here.
11/20/23 - Amendment 001: The BAA is amended to revise the submission window for white papers to December 1, 2023 12:00PM EST through December 5, 2023 12:00PM EST and to add the requirement for a Quad Chart attachment to the proposal, see attached Amendment 001.
________________________________________
Original posting:
White Papers (abstracts) Due Date: December 5, 2023 12:00 PM EST (Offerors are strongly encouraged to submit white papers before submitting a proposal). The submission window for white papers will open on November 27, 2023 12:00PM EST and close on December 5, 2023 12:00PM EST.
The Intelligence Advanced Research Projects Activity (IARPA) invests in high-risk/highpayoff research programs that have the potential to provide our nation with an overwhelming intelligence advantage. IARPA seeks to develop new capabilities to enable the safe adoption and use of generative AI technologies to greatly enhance the effectiveness and efficiency of the Intelligence Community (IC).
The goal of the BENGAL targeted super seedling is to understand LLM threat modes, quantify them and to find novel methods to correct threats and vulnerabilities or to work resiliently with imperfect models. IARPA seeks to develop and incorporate novel technologies to efficiently probe large language models to detect and characterize LLM threat modes and vulnerabilities. Performers will focus on one or more of the topic domains below, clearly articulate a taxonomy of threat modes within their domain of interest and develop technologies to serve as an analog to ‘virus scan’ software.
Update #3 ·
11/22/23 - Amendment 002: The BAA is amended for clarification, to revise the technical section, and to incorporate changes from Amendment 001. A full, updated BAA is attached. NOTE: The IDEAS system is not yet open for submission of white papers. It is anticipated that this will occur early next week. If there are any further delays we will issue a notice here.
11/20/23 - Amendment 001: The BAA is amended to revise the submission window for white papers to December 1, 2023 12:00PM EST through December 5, 2023 12:00PM EST and to add the requirement for a Quad Chart attachment to the proposal, see attached Amendment 001.
________________________________________
Original posting:
White Papers (abstracts) Due Date: December 5, 2023 12:00 PM EST (Offerors are strongly encouraged to submit white papers before submitting a proposal). The submission window for white papers will open on November 27, 2023 12:00PM EST and close on December 5, 2023 12:00PM EST.
The Intelligence Advanced Research Projects Activity (IARPA) invests in high-risk/highpayoff research programs that have the potential to provide our nation with an overwhelming intelligence advantage. IARPA seeks to develop new capabilities to enable the safe adoption and use of generative AI technologies to greatly enhance the effectiveness and efficiency of the Intelligence Community (IC).
The goal of the BENGAL targeted super seedling is to understand LLM threat modes, quantify them and to find novel methods to correct threats and vulnerabilities or to work resiliently with imperfect models. IARPA seeks to develop and incorporate novel technologies to efficiently probe large language models to detect and characterize LLM threat modes and vulnerabilities. Performers will focus on one or more of the topic domains below, clearly articulate a taxonomy of threat modes within their domain of interest and develop technologies to serve as an analog to ‘virus scan’ software.
Update #2 ·
11/20/23 - Amendment 001: The BAA is amended to revise the submission window for white papers to December 1, 2023 12:00PM EST through December 5, 2023 12:00PM EST and to add the requirement for a Quad Chart attachment to the proposal, see attached Amendment 001.
________________________________________
Original posting:
White Papers (abstracts) Due Date: December 5, 2023 12:00 PM EST (Offerors are strongly encouraged to submit white papers before submitting a proposal). The submission window for white papers will open on November 27, 2023 12:00PM EST and close on December 5, 2023 12:00PM EST.
The Intelligence Advanced Research Projects Activity (IARPA) invests in high-risk/highpayoff research programs that have the potential to provide our nation with an overwhelming intelligence advantage. IARPA seeks to develop new capabilities to enable the safe adoption and use of generative AI technologies to greatly enhance the effectiveness and efficiency of the Intelligence Community (IC).
The goal of the BENGAL targeted super seedling is to understand LLM threat modes, quantify them and to find novel methods to correct threats and vulnerabilities or to work resiliently with imperfect models. IARPA seeks to develop and incorporate novel technologies to efficiently probe large language models to detect and characterize LLM threat modes and vulnerabilities. Performers will focus on one or more of the topic domains below, clearly articulate a taxonomy of threat modes within their domain of interest and develop technologies to serve as an analog to ‘virus scan’ software.
Update #1 ·
White Papers (abstracts) Due Date: December 5, 2023 12:00 PM EST (Offerors are strongly encouraged to submit white papers before submitting a proposal). The submission window for white papers will open on November 27, 2023 12:00PM EST and close on December 5, 2023 12:00PM EST.
The Intelligence Advanced Research Projects Activity (IARPA) invests in high-risk/highpayoff research programs that have the potential to provide our nation with an overwhelming intelligence advantage. IARPA seeks to develop new capabilities to enable the safe adoption and use of generative AI technologies to greatly enhance the effectiveness and efficiency of the Intelligence Community (IC).
The goal of the BENGAL targeted super seedling is to understand LLM threat modes, quantify them and to find novel methods to correct threats and vulnerabilities or to work resiliently with imperfect models. IARPA seeks to develop and incorporate novel technologies to efficiently probe large language models to detect and characterize LLM threat modes and vulnerabilities. Performers will focus on one or more of the topic domains below, clearly articulate a taxonomy of threat modes within their domain of interest and develop technologies to serve as an analog to ‘virus scan’ software.
Files
Show all 12
On GovTribe
Work this opportunity
- Track it in your pipeline
- Find teaming partners
- Similar opportunities
- Ask GovTribe AI about this opportunity