Project Grant 2154973

This $1.2 million Project Grant from the National Science Foundation's Computer and Information Science and Engineering program will fund research at the University of Pittsburgh to expedite machine learning applications on multi-GPU infrastructure. Specifically, the university will uncover and address architectural bottlenecks in deep neural network executions on multi-GPU systems. Researchers will redesign translation lookaside buffer hierarchies and page table walks to reduce address translation bottlenecks for single-tenant and multi-tenant deep learning workloads. They will also investigate data movement overheads in data and model parallelism, proposing architecture-aware optimizations to mitigate these overheads. Additionally, the university will develop a runtime framework to enhance the programmability of multi-GPU resources and allow dynamic, automatic mapping of virtual to physical kernels. The three-year award period began on June 15, 2022 and will support this research through May 31, 2025.