Collaborative research: ccss: vision-structured photonic communication and sensing integration systems

The National Science Foundation Division of Electrical, Communications and Cyber Systems awarded Saint Louis University $223,759 on September 1, 2026, under the Engineering program (CFDA 47.041) to develop vision-structured photonic communication and sensing integration systems for underground operations.

The project addresses communication challenges in underground environments such as mining, infrastructure inspection, emergency response, and autonomous robotic exploration by enabling light-based communication systems to perceive their surroundings and adapt communication and sensing functions in real time. The resulting technologies are intended to support more reliable connectivity, improved awareness of human activity and environmental hazards, and safer operation of robots and personnel in complex underground spaces.

Work is performed in St. Louis, Missouri, with a period of performance from September 1, 2026, through August 31, 2029. The project integrates communication, sensing, robotics, and artificial intelligence into undergraduate and graduate courses, supports hands-on student research, and engages K-12 students and surrounding communities through interactive demonstrations. Open-source datasets, prototypes, benchmarks, and educational materials resulting from the project are intended to broaden access to outcomes and support future research in intelligent communication and cyber-physical systems.

Abstract

Reliable communication and environmental sensing are essential for safe underground operations, including mining, infrastructure inspection, emergency response, and autonomous robotic exploration. However, underground environments remain difficult for conventional wireless systems because rock, metallic equipment, irregular tunnels, dust, and moving machinery can weaken or disrupt radio signals. Light-based communication offers an attractive alternative because IT supports high data rates, avoids electromagnetic interference, and can use existing light sources such as helmet lamps, vehicle headlights, and ceiling fixtures. Yet current optical communication systems are highly vulnerable to blockage, misalignment, scattering, and changes in the surrounding environment because they operate with little understanding of physical structure. This project addresses this limitation by enabling light-based communication systems to perceive their surroundings and adapt communication and sensing functions in real time. The resulting technologies support more reliable connectivity, improved awareness of human activity and environmental hazards, and safer operation of robots and personnel in complex underground spaces. The project also advances education and workforce development by integrating communication, sensing, robotics, and artificial intelligence into undergraduate and graduate courses, supporting hands-on student research, and engaging k-12 students and surrounding communities through interactive demonstrations. Open-source datasets, prototypes, benchmarks, and educational materials broaden access to the project outcomes and support future research in intelligent communication and cyber-physical systems. This project establishes a new vision-structured photonic communication and sensing framework that connects three-dimensional scene understanding with optical signal propagation, communication control, and environmental sensing. The central scientific challenge is that existing optical systems cannot interpret how environmental structure affects signal propagation and therefore react only after communication quality has degraded. Visual mapping systems provide rich geometric information, but this information is not translated into channel models, communication decisions, or sensing functions. To overcome these challenges, the project pursues three integrated research directions. First, IT develops a vision-structured scene-channel integration framework that directly maps three-dimensional geometry and visual features to optical propagation behavior. The key innovation is to treat visual observations and photonic channels as two representations of the same underlying light interaction process, enabling the system to predict reflection, scattering, occlusion, link quality, and blockage risk before link failure occurs. Second, the project develops structure-aware communication mechanisms that use this environmental knowledge to select links, steer beams, allocate communication resources, choose routes, and adjust transmission parameters. These mechanisms move optical communication from reactive signal-based control toward proactive environment-aware adaptation. Third, the project develops a closed-loop framework for reciprocal communication and sensing, in which communication signals support environmental sensing and sensing outcomes guide communication control. The project evaluates these methods through simulation, laboratory communication testbeds, robotic platforms, and real-world experiments that reproduce blockage, dust, motion, and irregular underground geometry. The outcome is a general framework for resilient light-based communication and sensing in underground and other harsh environments. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are not planned for this award.