2a. Advanced Computing_Jenkins.pdf

This presentation details NASA's Ziggy, an open-source science data processing infrastructure developed by the NASA Advanced Supercomputing Division at Ames Research Center. The key problem addressed is that science data processing software is typically custom-built for each mission, incurring significant technical, cost, and schedule risks. Often developed by postdocs with limited software expertise, these pipelines tend to be brittle and introduce risks to mission science.

Ziggy offers a solution by providing a portable, scalable data processing framework that handles complex computational requirements across multiple NASA missions like Kepler, TESS, and Hyperion/EO-1. The system supports various computing environments (supercomputers, cloud, local servers), enables polyglot programming, and allows flexible pipeline configuration. Key features include automated logging, execution monitoring, error handling, and data accountability. The software is designed to be lightweight, easily deployable, and supports multiple programming languages including MATLAB, C++, Python, and Shell Script. Ziggy is freely available on GitHub and aims to reduce development time and complexity for scientific data processing pipelines.