Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI)
Closed Solicitation Posted
- Solicitation number
- ARPA-H-SOL-25-113
- Agency
- National Institutes of Health Department of Health and Human Services
- Responses due
- Set-aside
- No set-aside
Opportunity facts
- NAICS code
- 541715 Research and Development in the Physical, Engineering, and Life Sciences (except Nanotechnology and Biotechnology)
- Place of performance
- United States
- Points of contact
-
- PRECISE-AI Coordinator preciseai@arpa-h.gov
Notice details come from SAM.gov. Updated .
About this opportunity
The Department of Health and Human Services National Institutes of Health (NIH) through ARPA-H is seeking proposals for the Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) program. The solicitation aims to develop self-correction techniques for maintaining AI model performance in clinical healthcare settings, with a focus on creating tools that can autonomously monitor and improve AI Decision Support Tools (AI-DSTs). Key contract objectives include developing capabilities for continuous monitoring, degradation detection, root cause analysis, and bidirectional communication with clinicians. Potential proposers must submit a comprehensive Technical & Management Document, Task Description Document, Price/Cost Proposal, Model Agreement, and Administrative Requirements by January 15, 2025 at 5:00 PM ET. A Proposers' Day is scheduled for October 17, 2024, with a Q&A submission deadline of December 18, 2024.
The solicitation is not set aside for any specific business category and will be executed through Other Transaction Authority with multiple anticipated awards. The program will span four years, divided into three phases: Phase I (24 months) for prototyping, Phase II (12 months) for refinement and testing, and Phase III (12 months) for integration and transition. Proposers may submit for a single or multiple technical areas, which include automated surrogate ground truth label extraction, degradation detection, uncertainty quantification, core data infrastructure, and independent verification and validation. The place of performance is within the United States, targeting the development of an open-source repository of tools to maintain clinical AI model performance. Notably, FFRDC and U.S. Government entities are explicitly ineligible to serve as prime contractors or subcontractors in this opportunity.
Notice text
2 versions
Update #2 · Latest ·
December 18, 2024 - Amendment 01 posted. This amended provides a revised Attachment 1, which requests Key Personnel information on the cover page. All other documents remain unchanged.
The rapid advancement of artificial intelligence (AI) technologies is transforming healthcare by improving efficiencies, reducing costs, and enhancing health outcomes. This potential is evident with over 850 FDA-approved medical devices now incorporating AI functionalities, a tenfold increase from 2018 to 2023. However, the ability to ensure the ongoing safety and efficacy of these AI systems has not kept pace. The conventional safety testing approach relies heavily on pre-market testing, assuming that these initial results will predict long-term performance. However, pre-market results often fail to account for variations in operational processes and patient demographics, leading to unpredictable post-market performance that currently requires manual oversight by vendors. Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) aims to create a suite of self-correction techniques that make it possible to automatically maintain peak model performance of predictive AI components across diverse clinical settings. PRECISE-AI will advance novel approaches to optimally support clinician decision-making and scalably manage the performance of AI Decision Support Tools (AI-DSTs) after their commercial deployment. Key areas of innovation include continuous monitoring capabilities, degradation detection, root cause analysis, self-correction, and bidirectional communication with clinicians. This program will establish an open-source repository of tools to autonomously maintain the performance of clinical AI-DSTs while enhancing the interpretability and actionability of AI model outputs. The program will test these innovations in real-world settings to demonstrate measurable improvements in clinical decision-making. This program addresses the pressing need for continuous monitoring and updating of clinical AI models to ensure they remain effective and trustworthy over time.
Update #1 ·
The rapid advancement of artificial intelligence (AI) technologies is transforming healthcare by improving efficiencies, reducing costs, and enhancing health outcomes. This potential is evident with over 850 FDA-approved medical devices now incorporating AI functionalities, a tenfold increase from 2018 to 2023. However, the ability to ensure the ongoing safety and efficacy of these AI systems has not kept pace. The conventional safety testing approach relies heavily on pre-market testing, assuming that these initial results will predict long-term performance. However, pre-market results often fail to account for variations in operational processes and patient demographics, leading to unpredictable post-market performance that currently requires manual oversight by vendors. Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) aims to create a suite of self-correction techniques that make it possible to automatically maintain peak model performance of predictive AI components across diverse clinical settings. PRECISE-AI will advance novel approaches to optimally support clinician decision-making and scalably manage the performance of AI Decision Support Tools (AI-DSTs) after their commercial deployment. Key areas of innovation include continuous monitoring capabilities, degradation detection, root cause analysis, self-correction, and bidirectional communication with clinicians. This program will establish an open-source repository of tools to autonomously maintain the performance of clinical AI-DSTs while enhancing the interpretability and actionability of AI model outputs. The program will test these innovations in real-world settings to demonstrate measurable improvements in clinical decision-making. This program addresses the pressing need for continuous monitoring and updating of clinical AI models to ensure they remain effective and trustworthy over time.
Attachments
| File | Type | Posted |
|---|---|---|
| Amendment 01_Other Transaction Bundle Templates.zip | ZIP file | |
| ARPA-H-SOL-25-113.pdf | ||
| Other Transaction Bundle Templates.zip | ZIP file |
Notice history
| Notice | Type | Posted |
|---|---|---|
| Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) | Solicitation | |
| Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) | Special Notice | |
| Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) | Special Notice | |
| Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) | Pre-Solicitation |
On GovTribe
Work this opportunity on GovTribe
- Track it in your pipeline
- Find teaming partners
- Similar opportunities
- Ask GovTribe AI about this opportunity