SOW_-_HEP-CCE_Workflow_Framework_Evaluation.docx

Date

Statement of Work Test and Evaluation of Workflow Frameworks on HPC Systems Introduction

· The Computing and Data Sciences Directorate at Brookhaven National Laboratory (BNL) has developed portable workflow execution frameworks targeting high performance computing systems, integrating Globus Compute and PanDA workflow management systems. We are looking for a subcontractor with a background in high energy physics (HEP) simulations and/or data analysis to test and evaluate the workflow frameworks, in order to help us improve the frameworks to meet the needs of HEP experiments.

· Overall mission and other requirements: This work is part of the High Energy Physics Center for Computational Excellence (HEP-CCE) project (www.anl.gov/hep-cce), which aims to improve the utilization of high performance computing systems by HEP experiments through innovative algorithms and tools. Therefore, we strongly prefer applicants with the following backgrounds:

· PhD in experimental high energy physics

· Familiar with HEP simulation workflows, such as the LHC ATLAS simulation workflows

· Has a good track record of testing and documenting new software tools for scientific applications

· Has basic knowledge of HPC systems, including access, compilation and running, monitoring, data transfer and other related skills

· Good communication skills and can proactively report any issues discovered

Scope

· The contractor is expected to perform the following tasks:

· Set up HEP simulation software and related execution scripts on NERSC Permultter, and if time permits, on ALCF Polaris.

· Work with the BNL workflow team on integrating the HEP simulation software with the developed workflow framework based on Globus Compute and PanDA

· Run the HEP simulation software through the workflow framework

· Document the step-by-step findings of a successful execution, including how to set up the software environments and input parameters

· Evaluate the correctness and performance of the output by comparing the runs on the HPC systems with those from Grid sites.

· Report any issues promptly to the BNL workflow team.

· Suggest new features to be included in the next software release, and when new features are available, assist in testing and evaluating the new features.

· Help prepare any user guide, documentation or tutorials as needed related to the workflow framework.

· Help publicize the workflow framework in the HEP community through publications and presentations.

· As this work will require close communications with the BNL team, we strongly prefer local (within 50 miles from BNL) applicants who can travel to BNL for in-person meetings as needed.

· The duration of the work is expected to be completed between April 1, 2025 to December 31, 2025.

Schedule

· Milestones

· Successful execution of a HEP simulation workflow on NERSC Perlmutter without going through the workflow framework [1 month post award]

· Successful execution of the same workflow orchestrated by the new workflow framework [3 months post award]

· Performance evaluation of the framework based on correctness, run time and overhead of the environment set up [4 months post award]

· Generation of documentation and user guide for the framework, including step-by-step guide on how to set up the environments [6 months post award]

· Repeat the same evaluation on another HPC system, such as ALCF Polaris [8 months post award]

· A technical report summarizing the work performed by the contractor [9 months post award]

· The contractor is expected to attend weekly technical meetings related to the project.

· A mid-term review will be conducted to ensure progress.

image1.png