Attachment A - SOW (DRAFT).pdf

STATEMENT OF WORK

Project title: Vision-Guided Robotic Automation and Digital Twin Framework for Autonomous Experiments Date: 05-12-2026

Statement of Work:

• Introduction

Future synchrotron beamlines aim to operate with increasing levels of autonomy to improve efficiency, safety, and remote accessibility. A critical component of autonomous beamline operation is the ability to predict, evaluate, and validate actions before execution in the real beamline environment, thereby reducing risk and commissioning time.

This project will develop a vision-guided automation platform integrated with a digital twin and robotic simulation framework. Stereo camera systems will be deployed both in the laboratory and at beamline to provide real-time 3D perception. A robotic manipulator will be tested and validated in a dedicated laboratory using 3D-printed beamline environment mimics. A synchronized digital twin and simulation environment will be developed to predict robotic behavior, evaluate feasibility, and assess safety before real-world beamline deployment.

This work will provide a foundational subsystem for an autonomous beamline framework, enabling perception-driven decision-making, predictive safety assessment, and validated robotic workflows for spectroscopy experiments.

• Scope

1. Computer Vision Systems for Laboratory and Beamline Environments

- Deploy stereo camera systems in both laboratory and beamline environments.

- Develop 3D perception pipelines for reconstructing workspace geometry and detecting experimental components.

- Implement deep-learning-based object detection, classification, and pose estimation for:

• sample holders

• detectors

• robotic tools

• environmental fixtures

- Quantify perception uncertainty and propagate it to downstream robotic planning and safety evaluation.

2. Digital Twin Development

- Develop a digital twin representing:

DRAFT

• robotic kinematics and workspace geometry

• beamline-relevant experimental components

• mock beamline hardware in the laboratory environment

• Synchronize real-time perception data from stereo cameras with the digital twin to maintain an updated representation of the physical environment.

- Use the digital twin to visualize and analyze robotic operations and experimental configurations.

3. Robotic Simulation and Predictive Evaluation Framework

- Develop a robotic simulation environment mirroring the laboratory and beamline setups.

- Integrate robot kinematic models, environment geometry, and perception-derived object poses into the simulation.

- Implement pre-execution evaluation of robotic trajectories, including:

• reachability analysis

• collision prediction

• clearance margin estimation

• joint limit and singularity detection

- Compare simulated predictions with laboratory experimental outcomes to quantify simulation-to-reality discrepancies.

• Schedule

The anticipated timeline is as follows:

- Base Year

0-2 months:

▪ Hardware system setup

▪ Implement stereo depth reconstruction and point cloud generation

▪ Train and evaluate deep-learning models for object recognition

3-5 months:

▪ Commissioning for QAS beamline and Lab

6-9 months:

▪ Commissioning for TES beamline

9-12 months:

▪ Optimize algorithms based on commissioning results

▪ Paper preparation and submission

The student will work 20 hours per week with BNL and a weekly project meeting is expected.

Optional:

Year 1

• Deploy and calibrate stereo vision systems in laboratory and beamline environments

• Train and optimize deep-learning models for object recognition and pose estimation

• Develop perception-driven warning and collision-avoidance mechanisms

• Integrate perception outputs with robotic planning and safety evaluation workflows

Year2

• Develop digital twin geometry and robotic kinematic models

• Establish an initial robotic simulation environment for beamline operations

• Validate simulation predictions against laboratory experimental results

• Evaluate simulation-to-reality discrepancies and improve model accuracy

Signature/Date

Yonghua Du

Signature Name

Senior Physicist 6/2/2026

Title Date

Lu Ma

Signature Name

Physicist 6/2/2026

Title Date