This Project Grant award of $224,984 from the National Science Foundation's (NSF) Computer and Information Science and Engineering (CISE) program (CFDA 47.070) supports research at the College of William & Mary in Williamsburg, Virginia focused on developing automated checkpoint and restart capabilities for deep learning applications running on high-performance computing (HPC) clusters.
The key objectives of this 3-year project are to: 1) exploit properties of deep neural network models to enable efficient asynchronous and versioned checkpointing; 2) develop methods to schedule checkpointing tasks and I/O to utilize HPC system resources effectively; and 3) create a compilation system to automate the checkpointing process for diverse parallelization schemes used in deep learning workloads. This research aims to increase the resiliency, efficiency, and accessibility of training large deep learning models on shared HPC resources, benefiting a wide range of NSF-supported researchers and engineers.