Project Grant 2551892

The National Science Foundation Division of Computing and Communication Foundations awarded The Pennsylvania State University $292,743 on August 1, 2026, under the Computer and Information Science and Engineering program (CFDA 47.070) to develop artificial intelligence systems that understand physical scenes through structural abstraction rather than dense pixel-by-pixel reconstruction.

The project advances AI's capacity for common-sense physical reasoning by extracting semantically rich structural skeletons from images—simplified geometric representations using primitives such as planes and cylinders—enabling machines to reason about physical stability, cause and effect, and object interactions. This approach makes AI agents more robust and efficient in real-world environments and improves the reliability of autonomous systems including industrial robots and assistive devices for elderly or disabled individuals. The research will also enhance augmented and virtual reality performance on mobile devices.

The work spans a collaborative research partnership between Penn State University and the University of Louisville, with performance at Penn State's State College, Pennsylvania campus. The award runs through July 31, 2030. The project includes workforce development components training graduate and undergraduate students and integrating findings into university coursework across the collaborating institutions. Research thrusts focus on developing pipelines to extract structural skeletons from single-view images using robust primitive fitting from monocular depth estimates, grounding semantic predictions with large vision models, and advancing physical reasoning capabilities in AI systems.