Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI)
Closed Pre-Solicitation Posted
A newer solicitation was posted. See the latest solicitation from .
- Solicitation number
- ARPA-H-SOL-25-113
- Agency
- National Institutes of Health Department of Health and Human Services
- Responses due
- Set-aside
- No set-aside
Opportunity facts
- NAICS code
- 541715 Research and Development in the Physical, Engineering, and Life Sciences (except Nanotechnology and Biotechnology)
- Place of performance
- United States
- Points of contact
-
- PRECISE-AI Coordinator preciseai@arpa-h.gov
Notice details come from SAM.gov. Updated .
About this opportunity
The Department of Health and Human Services' National Institutes of Health (NIH) is seeking proposals for the Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) program. This program aims to create self-correction techniques that can automatically maintain the peak performance of predictive AI components across diverse clinical settings. Key areas of innovation include continuous monitoring, degradation detection, root cause analysis, self-correction, and improved communication with clinicians. The program will establish an open-source repository of tools to autonomously maintain the performance of clinical AI Decision Support Tools (AI-DSTs) and enhance their interpretability and actionability. Proposals can address individual technical areas or combinations of areas related to automated ground truth extraction, degradation detection and self-correction, uncertainty quantification and clinician performance improvement, core data infrastructure, and independent verification and validation. Proposals are due on January 15, 2025, and the final solicitation will be posted at a later date.
This pre-solicitation notice does not identify any set-aside designations or potential incumbent contractors. The solicitation indicates that multiple awards are anticipated, but does not provide any details on possible award values or budget ranges. The program is structured in three phases over four years, with progression to subsequent phases dependent on performance against specified milestones and metrics.
Notice text
3 versions
Update #3 · Latest ·
October 28, 2024: Revised DRAFT posted. Proposers Day slides added.
September 5, 2024: Appendix A added
This is a DRAFT program solicitation. A final solicitation will be posted at a later date. ARPA-H intends to hold a Proposers' Day; details will be provided as they become available. Interested parties are encouraged to submit questions on the contents of this draft solicitation. No awards will be made as a result of this posting.
The rapid advancement of artificial intelligence (AI) technologies is transforming healthcare by improving efficiencies, reducing costs, and enhancing health outcomes. This potential is evident with over 850 FDA-approved medical devices now incorporating AI functionalities, a tenfold increase from 2018 to 2023. However, the ability to ensure the ongoing safety and efficacy of these AI systems has not kept pace. The conventional safety testing approach relies heavily on pre-market testing, assuming that these initial results will predict long-term performance. However, pre-market results often fail to account for variations in operational processes and patient demographics, leading to unpredictable post-market performance that currently requires manual oversight by vendors. Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) aims to create a suite of self-correction techniques that make it possible to automatically maintain peak model performance of predictive AI components across diverse clinical settings. PRECISE-AI will advance novel approaches to optimally support clinician decision-making and scalably manage the performance of AI Decision Support Tools (AI-DSTs) after their commercial deployment. Key areas of innovation include continuous monitoring capabilities, degradation detection, root cause analysis, self-correction, and bidirectional communication with clinicians. This program will establish an open-source repository of tools to autonomously maintain the performance of clinical AI-DSTs while enhancing the interpretability and actionability of AI model outputs. The program will test these innovations in real-world settings to demonstrate measurable improvements in clinical decision-making. This program addresses the pressing need for continuous monitoring and updating of clinical AI models to ensure they remain effective and trustworthy over time.
Update #2 ·
September 5, 2024: Appendix A added
This is a DRAFT program solicitation. A final solicitation will be posted at a later date. ARPA-H intends to hold a Proposers' Day; details will be provided as they become available. Interested parties are encouraged to submit questions on the contents of this draft solicitation. No awards will be made as a result of this posting.
The rapid advancement of artificial intelligence (AI) technologies is transforming healthcare by improving efficiencies, reducing costs, and enhancing health outcomes. This potential is evident with over 850 FDA-approved medical devices now incorporating AI functionalities, a tenfold increase from 2018 to 2023. However, the ability to ensure the ongoing safety and efficacy of these AI systems has not kept pace. The conventional safety testing approach relies heavily on pre-market testing, assuming that these initial results will predict long-term performance. However, pre-market results often fail to account for variations in operational processes and patient demographics, leading to unpredictable post-market performance that currently requires manual oversight by vendors. Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) aims to create a suite of self-correction techniques that make it possible to automatically maintain peak model performance of predictive AI components across diverse clinical settings. PRECISE-AI will advance novel approaches to optimally support clinician decision-making and scalably manage the performance of AI Decision Support Tools (AI-DSTs) after their commercial deployment. Key areas of innovation include continuous monitoring capabilities, degradation detection, root cause analysis, self-correction, and bidirectional communication with clinicians. This program will establish an open-source repository of tools to autonomously maintain the performance of clinical AI-DSTs while enhancing the interpretability and actionability of AI model outputs. The program will test these innovations in real-world settings to demonstrate measurable improvements in clinical decision-making. This program addresses the pressing need for continuous monitoring and updating of clinical AI models to ensure they remain effective and trustworthy over time.
Update #1 ·
This is a DRAFT program solicitation. A final solicitation will be posted at a later date. ARPA-H intends to hold a Proposers' Day; details will be provided as they become available. Interested parties are encouraged to submit questions on the contents of this draft solicitation. No awards will be made as a result of this posting.
The rapid advancement of artificial intelligence (AI) technologies is transforming healthcare by improving efficiencies, reducing costs, and enhancing health outcomes. This potential is evident with over 850 FDA-approved medical devices now incorporating AI functionalities, a tenfold increase from 2018 to 2023. However, the ability to ensure the ongoing safety and efficacy of these AI systems has not kept pace. The conventional safety testing approach relies heavily on pre-market testing, assuming that these initial results will predict long-term performance. However, pre-market results often fail to account for variations in operational processes and patient demographics, leading to unpredictable post-market performance that currently requires manual oversight by vendors. Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) aims to create a suite of self-correction techniques that make it possible to automatically maintain peak model performance of predictive AI components across diverse clinical settings. PRECISE-AI will advance novel approaches to optimally support clinician decision-making and scalably manage the performance of AI Decision Support Tools (AI-DSTs) after their commercial deployment. Key areas of innovation include continuous monitoring capabilities, degradation detection, root cause analysis, self-correction, and bidirectional communication with clinicians. This program will establish an open-source repository of tools to autonomously maintain the performance of clinical AI-DSTs while enhancing the interpretability and actionability of AI model outputs. The program will test these innovations in real-world settings to demonstrate measurable improvements in clinical decision-making. This program addresses the pressing need for continuous monitoring and updating of clinical AI models to ensure they remain effective and trustworthy over time.
Attachments
| File | Type | Posted |
|---|---|---|
| ARPA-H-SOL-25-113_DRAFT v2.pdf | ||
| PRECISE-AI_ProposersDay Slides.pdf | ||
| ARPA-H-SOL-25-113_Appendix A.docx | DOCX document | |
| OT Bundle of Attachments.zip | ZIP file | |
| ARPA-H-SOL-25-113_DRAFT.pdf |
Notice history
| Notice | Type | Posted |
|---|---|---|
| Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) | Solicitation | |
| Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) | Special Notice | |
| Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) | Special Notice | |
| Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) | Pre-Solicitation |
On GovTribe
Work this opportunity on GovTribe
- Track it in your pipeline
- Find teaming partners
- Similar opportunities
- Ask GovTribe AI about this opportunity