Project Grant 2224054

This three-year, $532,241 Project Grant from the National Science Foundation's Division of Computer and Network Systems, under the Computer and Information Science and Engineering program (CFDA 47.070), will support the development of scalable algorithms, systems, and infrastructures for graph neural network training. The University of Massachusetts will develop a novel "split parallelism" training paradigm to transparently scale graph neural network training to large-scale graphs and models across distributed and multi-GPU systems. This approach is tailored to graph neural network bottlenecks. The University will also create a fragment-based in-GPU sampling approach to maximize data locality and scalability for the performance-critical task of graph sampling in graph neural network training. These techniques aim to enable practitioners to more easily develop large, expressive models without workarounds for current scalability limitations. Findings will be integrated into popular open-source tools like the Deep Graph Library to transfer the technology. The award also supports related graduate and undergraduate student training to emphasize demographic diversity.