Crii: CSR: RUI: novel approaches for task graph scheduling algorithm design, evaluation, and comparison

Loyola Marymount University received a $150,000 Project Grant from the National Science Foundation (NSF) Division of Computer and Network Systems under the Computer and Information Science and Engineering (CISE) program (CFDA 47.070). The award, effective August 1, 2025, through July 31, 2027, supports the design, evaluation, and comparison of task graph scheduling algorithms for distributed computing systems. The project extends an open-source framework to integrate with widely used distributed computing platforms including Pegasus, Parsl, and Docker, enabling empirical evaluation of scheduling algorithms on real applications and systems. The work incorporates constraint-based scheduling and scheduling in stochastic environments, leveraging machine learning to develop intelligent scheduling approaches that improve task execution efficiency, resource utilization, and system reliability across scientific workflows, machine learning pipelines, and Internet of Things (IoT) applications.

The project also delivers significant educational and workforce development outcomes by providing hands-on undergraduate research opportunities at this primarily undergraduate institution. By strengthening the pipeline of future computing researchers, the initiative addresses national workforce development needs while advancing national research infrastructure and contributing to the development of robust, efficient, and scalable computing systems. The research outcomes and enhanced open-source framework will be disseminated to support the broader scientific and computing research communities.

Abstract

The task scheduling problem is fundamental to distributed computing and has thus been studied extensively over the past several decades. Better task scheduling approaches will improve how computational tasks execute on distributed computing systems, which support a vast array of important domains including scientific workflows, machine learning pipelines, and internet of things (IOT) applications. Efficient task scheduling helps these systems run faster, use resources more effectively, and deliver more reliable performance. Although many scheduling algorithms exist, IT is not well understood how they perform across different types of real-world scenarios, or in which kinds of scenarios they may perform poorly. This work builds upon an open-source framework to design, evaluate, and compare scheduling algorithms under realistic conditions to better understand their capabilities. With this resulting insight, the work explores new task scheduling approaches that leverage the strengths of prior approaches and employs machine learning to enable intelligent scheduling. The project provides hands-on undergraduate research opportunities at a primarily undergraduate institution, strengthening the pipeline of future computing researchers and helping address national workforce development needs. These efforts support national research infrastructure, scientific progress, and the development of robust, efficient, and scalable computing systems for the public good. This project extends an open-source framework to integrate with widely used distributed computing platforms such as pegasus, parsl, and docker, enabling empirical evaluation of scheduling algorithms on real applications and real systems. This work also adds support for constraint-based scheduling and scheduling in stochastic environments (both of which more accurately reflect the uncertainties and constraints common in practice). The project introduces methods for automatically designing hybrid scheduling algorithms by identifying and combining the strengths of existing ones using adversarial analysis. IT also explores how adversarially generated problem instances can be used to improve the training of machine learning-based schedulers by enriching their training data with informative, diverse examples. By providing standardized tools and methodologies, this work promotes reproducibility, encourages innovation, and empowers a broad range of researchers to better understand and improve task scheduling in heterogeneous distributed environments. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are not planned for this award.