Federal Project Grant Award Summary

The National Institute of Environmental Health Sciences (NIEHS), under the Medical Library Assistance program (CFDA 93.879), awarded The Johns Hopkins University a $588,676 Project Grant (awarded July 14, 2026, with completion targeted for April 30, 2031) to develop automated quantitative dataset audit methods for identifying bias, shortcuts, and robustness risks in data-driven healthcare artificial intelligence (AI) and machine learning (ML) applications. The project addresses a critical gap in AI/ML validation: while controlled testing demonstrates adequate algorithm performance, real-world deployment often yields 20% or greater performance degradation. The core deliverables include development of an automated audit framework and testbeds in medical image analysis designed to detect dataset biases independent of specific ML/AI models—including identification of non-clinically relevant feature shortcuts and imaging condition variations—rather than relying solely on post-hoc trained model evaluation.

This initiative directly supports the Medical Library Assistance program's objective of conducting discrete scientific research projects using biomedical informatics and data science methods to improve human health. By providing Software as a Medical Device (SAMD) stakeholders with explicit, model-independent dataset assessment tools, the project aims to quantify and measure how effectively ML/AI methods address undesirable biases prior to deployment, thereby reducing erroneous analyses, statistical fallacies, and representational errors in healthcare AI systems and enhancing the safety and efficacy of FDA-approved ML/AI medical devices.