Automated pet report generation with visual grounding

Active Grant Awarded

Award ID
R01EB038264
Recipient
Awarding agency
Obligated
$595,791

Award facts

Completion date 5% complete
Place of performance
Madison, WI 53715, USA

Award details come from USAspending.gov.

About this award

The Department of Health and Human Services, National Institutes of Health, National Institute of Biomedical Imaging and Bioengineering awarded Grant R01EB038264 to the University of Wisconsin System, through its University of Wisconsin - Madison division, with $595,791 obligated on September 9, 2026, under Assistance Listing 93.286, Discovery and Applied Research for Technological Innovations to Improve Human Health. The program supports research on the discovery, development, validation, and application of biomedical imaging and bioengineering technologies.

The project builds an automated PET report generation framework for nuclear medicine, aimed at relieving physicians of time-consuming reporting tasks. Its three components are a federated-learning vision-language foundation model, described as the field's first multi-center model of its kind; PETAR, a longitudinally aware detect-and-describe pipeline that identifies all findings, assesses interval change, and generates a visually grounded, structured list of findings; and an LLM synthesis module that generates a background section from the electronic medical record, consolidates the findings, and writes a final impression in the reading physician's own style.

Deliverables include public release of open-source foundation models and the first large-scale, labeled multimodal PET dataset. The period of performance ends May 31, 2030, and the place of performance is Madison, WI 53715, USA.

Description

Abstract
The increasing use of positron emission tomography (pet) imaging, particularly with the rise of theranostics, is creating a significant challenge for the nuclear medicine field. While pet scan volumes are growing by ~10% annually, the workforce of trained physicians is not keeping pace. This growing disparity places immense pressure on physicians, potentially compromising the quality of care and leading to delayed diagnoses. Much of a physician's time is spent on tasks ancillary to the overall interpretative task, and solutions that can improve physician efficiency by automating these tasks are needed. This project will develop an innovative end-to-end framework to solve this. First, we will use federated learning to develop nuclear medicine's first multi-center vision-language foundation model, overcoming the problem of data scarcity. Second, we will build a novel, longitudinally-aware detect-and-describe pipeline (PETAR) that identifies all findings, assesses interval change, and generates a visually-grounded, structured list of findings. Finally, an LLM synthesis module will complete the full report by generating a background section from the electronic medical record, consolidating the findings, and creating a final impression in the reading physician's own style. Together, this framework will relieve physicians of time- consuming tasks, allowing them to focus on their core expertise. This project will establish a new paradigm for multimodal AI in low-volume imaging. The public release of our open-source foundation models and the first large-scale, labeled multimodal pet dataset will be a major contribution to the field. This will catalyze a new wave of research, providing the entire community with the open-science tools needed to solve this critical challenge in nuclear medicine.

Quick answers

Who received award R01EB038264?

R01EB038264 was awarded to University Of Wisconsin System by National Institute of Biomedical Imaging and Bioenginering (National Institutes of Health) on September 9, 2026.

How much is award R01EB038264 worth?

$595,791 has been obligated on R01EB038264.

On GovTribe

See this award's full record

  • Every transaction and subaward
  • The recipient's other awards and teaming partners
  • Similar grant awards
  • Ask GovTribe AI about this award