This $225,000 Project Grant awarded by the National Science Foundation (NSF) under the Computer and Information Science and Engineering (CFDA 47.070) program supports the CROPDL research project at Florida State University (FSU). The key objectives of CROPDL are to develop efficient application-level checkpoint and restart capabilities to enable more resilient, faster, and higher resource-utilized deep learning training on shared high-performance computing (HPC) clusters.
The project aims to leverage the unique properties of deep learning workloads, such as limited communication patterns and malleable execution, to enable asynchronous versioned checkpointing and content-based data reduction techniques. Additionally, the research focuses on scheduling and I/O optimizations to utilize current and emerging HPC system resources effectively during checkpointing. The project also plans to automate checkpoint/restart functionality through a compilation system based on deep learning workload computational graphs. This work supports NSF's goal of advancing cyberinfrastructure and computational capabilities to drive scientific discovery across diverse domains.