Project Grant 2401246

This $131,959 Project Grant awarded by the National Science Foundation's (NSF) Computer and Information Science and Engineering (CISE) program supports research conducted by Rutgers, The State University to develop new deep learning training methods that can efficiently scale to utilize high-performance computing (HPC) systems. The key goals are to: 1) Explore techniques like second-order information approximation, computation-communication tradeoffs, and data compression to enhance the speed and scalability of deep learning optimization without sacrificing model performance; 2) Apply the new algorithms and systems to a range of scientific applications using deep learning; and 3) Release an open-source implementation of the proposed approaches. This work aims to enable faster training of larger deep learning models, which will advance research in fields like drug design, environmental monitoring, and fusion energy. The project also includes educational components such as integrating the tools into a new deep learning systems engineering course and recruiting underserved students to apply them to scientific problems.