ARPA-H-SN-25-116.pdf

Special Notice:

ARPA-H Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) Hybrid Proposers’ Day

ARPA-H-SOL-25-113

ADVANCED RESEARCH PROJECTS AGENCY FOR HEALTH

RESILIENT SYSTEMS OFFICE

Special Notice (SN): ARPA-H-SN-25-116

OPPORTUNITY Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI)

EVENT Hybrid Proposers’ Day

EVENT DATE October 17, 2024

PROGRAM WEBSITE https://arpa-h.gov/research-and-funding/programs/precise-ai

REGISTRATION DEADLINE October 14 at 5 PM ET

REGISTRATION WEBSITE https://solutions.arpa-h.gov/PRECISE-AI

REGISTRATION POC PRECISE-AI@arpa-h.gov

TECHNICAL POC Berkman Sahiner, Ph.D. Program Manager, Resilient Systems Office

PROPOSERS’ DAY

The Advanced Research Projects Agency for Health (ARPA-H) is hosting a Proposers’ Day in support of the Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) Program. The Proposers' Day will be a one-day hybrid event on October 17, 2024, beginning around 9:30 AM to approximately 5 PM Central Time (CT) in-person (the virtual event will end around 12 PM CT). Advance registration is required for in person and virtual attendance. The registration deadline for Proposers’ Day is October 14 at 5 PM Eastern Time (ET). The PRECISE-AI Program Solicitation ARPA-H-SOL-25-113 is available on https://sam.gov and is linked directly to from the Program Website https://arpa-h.gov/research-and-funding/programs/precise-ai.

PROPOSERS’ DAY GOALS

1. Introduce the ARPA-H PRECISE-AI program goals and vision to the research community;

2. Clarify the logistics of an ARPA-H program as well as PRECISE-AI specific objectives and milestones;

3. Review the PRECISE-AI program deliverables;

4. Encourage and promote teaming among potential Performers; and

5. Collect relevant questions and feedback on the program. Questions and answers will be maintained on a Q&A document at https://arpa-h.gov/research-and-funding/programs/precise-ai

PROPOSERS’ DAY INFORMATION

Registration Information

In-person attendance is limited by the venue capacity and early registration is strongly recommended.

In-person registration may close prior to the registration deadline due to the venue capacity being met.

https://arpa-h.gov/research-and-funding/programs/precise-ai https://solutions.arpa-h.gov/PRECISE-AI mailto:PRECISE-AI@arpa-h.gov https://sam.gov/ https://arpa-h.gov/research-and-funding/programs/precise-ai https://arpa-h.gov/research-and-funding/programs/precise-ai

Should demand exceed venue capacity, organizations may be asked to limit the number of participants attending in-person. The determination of what constitutes an organization will be made by the ARPA- H Program Manager. Registration for the virtual event is unlimited. Any changes to the registration deadline will be posted on the registration website at https://solutions.arpa-h.gov/PRECISE-AI.

Participants must register in advance through the registration website for in-person or virtual attendance. The registration deadline for the PRECISE-AI Program Proposers’ Day is October 14 at 5 PM ET. There will be no same-day registration. In-person registration must be confirmed by ARPA- H. All attendees, regardless of participation status will receive an email with the webinar link prior to event kick-off. The event is closed to the general public and media.

Event Format

The ARPA-H PRECISE-AI Proposers’ Day will be a hybrid one-day event. It will be hosted in-person in Madison, WI and virtually via Zoom for Government.

The PRECISE-AI Proposers’ Day will include presentations by ARPA-H, including technical presentations covering the PRECISE-AI Program as well as informational sessions during which questions from participants will be answered. In-person attendees will also have the opportunity to participate in a poster session to highlight the organization’s relevant capabilities to foster potential teaming arrangements. Please review the individual event element sections following the DRAFT agenda for more information.

Proposers’ Day Notional DRAFT Agenda

(Agenda subject to change. All registered attendees will be receiving a final agenda prior to event kick-off.)

9:30 am 12:00 am Introductions, ARPA-H goals, program overview, translation into real-world products, perspectives on AI-DSTs

11:55 am 1:30 pm Lunch On Your Own 1:30 pm 2:00 pm Q&A Session 2:00 pm 5:00 pm Sidebar Discussions 2:00 pm 2:15 pm Poster Setup 2:15 pm 4:00 pm Poster Session

1:1 Sidebar Meetings

Sidebar sessions will be hosted for in-person attendees only. Requests for a sidebar can be made via the registration website, and requests should be received no later than the registration cutoff date. Only one (1) sidebar will be allowed per organization and will be available on a first-come basis. Sidebar discussions will be limited to five (5) minutes. Due to limited slots, registration does not guarantee a sidebar appointment. Confirmation of sidebar slots will be sent by the PRECISE-AI team prior to Proposers’ Day.

Posters

During registration, interested attendees and/or proposers are invited to request space for one poster (to fit within a space measuring 3.75 feet wide by 3.75 feet high - landscape layout preferred, on paper or cloth) that summarizes their interests and capabilities relevant to the PRECISE-AI program. During this session, proposers will be split in 5 categories based on TA alignment (TA1: Automated Surrogate Ground Truth

Label Extraction; TA2: Degradation Detection & Self-Correction; TA3: Quantify Uncertainty & Improvement of Clinician Performance; TA4: Core Data Infrastructure; TA5: Independent Verification and Validation), and proposers will have the opportunity to present their work and answer questions from parties interested in exploring a collaboration. Posters will be accepted on a first-come, first-served basis distributed across TAs, until the space constraint is reached. Interested attendees must indicate their intention to present a poster during registration. Poster presentations are restricted to those attending the meeting in-person.

PECISE-AI PROGRAM DESCRIPTION

The rapid advancement of artificial intelligence (AI) technologies is transforming healthcare by improving efficiencies, reducing costs, and enhancing health outcomes. This potential is evident with over 950 FDA-approved medical devices now incorporating AI functionalities, a tenfold increase from 2018 to 2023.

However, the ability to ensure the ongoing safety and efficacy of these AI systems has not kept pace. The conventional safety testing approach relies heavily on pre-market testing, assuming that these initial results will predict long-term performance. However, pre-market results often fail to account for variations in operational processes and patient demographics, leading to unpredictable post-market performance that currently requires manual oversight by vendors. Performance and Reliability Evaluation for Continuous modIfications and uSEability of AI (PRECISE-AI) aims to create a suite of self-correction techniques that make it possible to automatically maintain peak model performance of predictive AI components across diverse clinical settings. PRECISE-AI will advance novel approaches to optimally support clinician decision-making and scalability manage the performance of AI Decision Support Tools (AI-DSTs) after their commercial deployment. Key areas of innovation include continuous monitoring capabilities, degradation detection, root cause analysis, self-correction, and bidirectional communication with clinicians.

This program will establish an open-source repository of tools to autonomously maintain the performance of clinical AI-DSTs while enhancing the interpretability and actionability of AI model outputs. The program will test these innovations in real-world settings to demonstrate measurable improvements in clinical decision-making. This program addresses the pressing need for continuous monitoring and updating of clinical AI models to ensure they remain effective and trustworthy over time.

SUPPLEMENTAL INFORMATION

Teaming Profiles

Interested parties are invited to submit a ‘teaming profile’ by navigating to the teaming page https://arpa-h.gov/research-and-funding/programs/precise-ai/teaming. The teaming profile will describe the technical competencies, team capabilities, team composition, research areas of interest, unique facilities, and other capabilities, as they relate to the program, and desired technical/other competencies sought from other potential team partners. The profile will include, at a minimum:

1. Contact information, to include name, organization, email, telephone number, mailing address, and website;

2. Brief description of the proposer’s technical competencies;

3. Desired technical competencies and facilities from other potential team members, if applicable.

https://arpa-h.gov/research-and-funding/programs/precise-ai/teaming https://arpa-h.gov/research-and-funding/programs/precise-ai/teaming

Specific content, communications, networking, and team information are the sole responsibility of participants. ARPA-H does not endorse any participating organization or exercise any responsibility for improper dissemination of the team profiles.

Administrative

All administrative and technical questions regarding the ARPA-H PRECISE-AI Proposers’ Day event should be directed to the PRECISE-AI FAQs Page which is linked on the Program Website. Please refer to the Proposers’ Day Special Notice number (ARPA-H-SN-25-116) in all correspondence.

This special notice is issued solely for information and potential new project planning purposes; the notice does not constitute a formal solicitation for proposals nor solution summaries. Any proposals or solution summaries received in response to this notice will be discarded.

Attendance at Proposers’ Day is voluntary and is not required to submit a proposal or solution summary to subsequent ARPAH PRECISE-AI research solicitation (if any) on this topic. ARPA-H will not reimburse costs incurred in responding to this Special Notice. Respondents are advised that ARPA-H is under no obligation to acknowledge receipt of any information received or provide feedback to respondents with respect to any information submitted under this Special Notice.

https://solutions.arpa-h.gov/SignIn?ReturnUrl=%2FAsk-A-Question%2F