Cici: ipaai: a data provenance framework for medical machine learning research

Award: Data Provenance Framework for Medical Machine Learning Research

Funding Agency: National Science Foundation (NSF), Office of Integrative Activities

Program: Integrative Activities (CFDA 47.083)

Total Obligated Funding: $900,000

Award Date: November 1, 2025

Performance Period: November 1, 2025 – October 31, 2028

Prime Awardee: University of California, Los Angeles

This project will develop the first comprehensive end-to-end data provenance framework for medical artificial intelligence (AI) systems that enables tracing, auditing, and efficient removal of patient data from trained models. The framework addresses critical gaps in medical AI reliability and trustworthiness by providing tools to identify data integrity problems, trace the origin of training datasets, detect tampering or corruption, and remove compromised or sensitive patient records without requiring full model retraining. The deliverables include three core research thrusts: (1) automated methods to detect when public machine learning models were trained on corrupt datasets; (2) efficient logging mechanisms to track dataset and model usage in medical research workflows; and (3) machine unlearning algorithms that remove sensitive data points without degrading model performance. Additionally, the award supports development of open-source tools and educational coursework to prepare the next generation of medical and computer scientists in trustworthy AI development.

The University of Colorado Colorado Springs, through a sub-grant award (December 2, 2025), will contribute expertise in healthcare data workflow analysis and modeling, the design of lightweight provenance logging mechanisms, and the development of tools to detect malicious compromise or unintentional errors in healthcare data pipelines. The sub-awardee's principal investigator will lead efforts across all three years to analyze clinical data workflows, model complex dependencies in healthcare datasets, implement efficient logging systems, and build detection tools that identify sources of data compromise for removal from AI models. Educational materials will be developed and integrated into university coursework throughout the project period.

Abstract

Artificial intelligence (AI) systems that read clinical notes and medical images promise earlier diagnoses, personalized treatments, and lower costs. However, these systems face critical challenges that threaten their reliability and ethical use. Data integrity problems, such as mistakes or tampering, can distort models and endanger patient care. In addition, patient data may be withdrawn due to revoked consent or legal obligations. There is no reliable way to see where a medical model's training data came from, whether that data was tampered with, or how to delete patient records effectively and efficiently from a model. This project will create the first end-to-end provenance framework for medical AI that enables tracing, auditing, and, when necessary, removing data efficiently and efficiently. The results will improve patient privacy, reliability of medical AI, and provide open-source tools for trustworthy AI. Building this framework is challenging, as medical datasets are unstructured, multimodal, dynamic, and come from many providers. Further, removing tainted data can force a full model retrain and thus harm model performance. To address these challenges, this research has three thrusts: (1) automated inferences of public machine learning (ML) models that were trained on corrupt datasets, (2) efficient logging of datasets and ML models usage in medical research workflows, and (3) efficient machine unlearning to remove compromised or sensitive data points without retraining. This project advances the foundations of secure medical data provenance, machine unlearning, and provides open-source tools and coursework that prepare the next generation of medical and computer scientists to build trustworthy AI. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are planned for this award.