III: exploratory volume visualization via explicit gaussian splatting and natural language interaction

The National Science Foundation Division of Computing and Communication Foundations awarded the University of Notre Dame DU Lac $700,000 on October 1, 2026, under the Computer and Information Science and Engineering program (CFDA 47.070) to develop a framework for visualization synthesis based on Gaussian splatting that enables faster, more interactive exploration of complex three-dimensional scientific datasets such as weather systems, fluid flows, medical scans, and engineering simulations.

The research investigates four interconnected directions: editable Gaussian representations for inverse and interactive volume rendering; deformable Gaussian models for segmentation and feature tracking in time-varying scientific data; textured Gaussian representations for expressive visual editing and stylization of volumetric scenes; and large language model agents supporting natural language interaction with scientific visualizations. By enabling machine learning models to generate visual representations directly without repeatedly processing original data, the project aims to accelerate scientific discovery, broaden participation in data-intensive research, and improve communication of scientific results to students, educators, and the public.

Educational activities include curriculum development, student training, outreach to K-12 educators, dissemination of open-source software, and public-facing visualization experiences to prepare the next generation of the artificial intelligence and scientific computing workforce.

Performance occurs at Notre Dame, Indiana, with a period of performance from October 1, 2026, through September 30, 2030. The award is classified as a Project Grant, with $700,000 obligated at the time of award.

Abstract

Scientific data generated by simulations, experiments, and observations are growing so rapidly that researchers often struggle to efficiently explore and understand them. Volume visualization, a key technology for examining complex three-dimensional data such as, weather systems, fluid flows, medical scans, and engineering simulations, typically requires access to large datasets and computationally-intensive processing. This project investigates a new paradigm called visualization synthesis, in which machine learning models learn how scientific visualizations appear and generate new visual representations directly, without repeatedly processing the original data. By enabling faster, more interactive, and more accessible exploration of complex datasets, the project has the potential to accelerate scientific discovery, broaden participation in data-intensive research, and improve the communication of scientific results to students, educators, and the public. Educational activities associated with the project include curriculum development, student training, outreach to k-12 educators, dissemination of open-source software, and public-facing visualization experiences that help prepare the next generation of the artificial intelligence and scientific computing workforce. This project develops a general framework for visualization synthesis based on gaussian splatting, an emerging machine learning representation that enables efficient modeling and rendering of complex scenes. The research investigates four interconnected directions: editable gaussian representations for inverse and interactive volume rendering; deformable gaussian models for segmentation and feature tracking in time-varying scientific data; textured gaussian representations for expressive visual editing and stylization of volumetric scenes; and large language model agents that support natural language interaction with visualizations. The team will design new algorithms, benchmark datasets, and evaluation methodologies to measure representation compactness, convergence speed, rendering quality, flexibility, and usability. Outcomes of the project include open-source software libraries, standardized benchmarks for language-guided visualization systems, and immersive virtual reality applications that combine visualization synthesis, scene stylization, and conversational interfaces. The resulting advances will establish foundational methods for machine learning-driven scientific visualization and enable new forms of interactive analysis, education, and public engagement. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are not planned for this award.