ERI: RUI: controlling and designing soft material robots for mixed reality applications

The National Science Foundation's Division of Civil, Mechanical, and Manufacturing Innovation awarded $174,934 to Williams College under the Engineering program (CFDA 47.041) on September 1, 2025, through an Engineering Research Initiation (ERI) grant for Research in Undergraduate Institutions (RUI). This two-year project, concluding August 31, 2027, will develop fundamental research and a principled framework for controlling and designing soft material robots capable of producing mixed reality experiences that combine physical and virtual elements. The research will deliver optimization-based control methodologies that enable real-time coordination between physical soft robots and corresponding virtual avatars, addressing a critical gap in existing inverse kinematics solutions that currently cannot maintain physical-virtual coherence during user interaction.

The project will generate products and research outcomes applicable to high-impact applications including surgical training simulators, educational demonstrations, and broader applications across healthcare, manufacturing, education, and agriculture sectors. Additionally, the award provides significant educational value by engaging undergraduate students at Williams College in novel robotics research at the intersection of engineering and computer science, advancing both scientific knowledge and workforce development in advanced robotics technology.

Abstract

This engineering research initiation (ERI) research in undergraduate institutions (RUI) grant will support research to contribute new knowledge related to a class of robots, promoting both the progress of science and advancing national prosperity. Soft robots are robots with bodies made from soft materials like rubbers and foams. Soft robots are a natural choice for applications requiring safe, close interactions between humans and robots. One particular class of applications that promises to benefit from soft robots are mixed reality applications, which combine physical and virtual elements to create immersive experiences that users can both touch and see. However, there currently exists no principled framework for controlling and designing soft robots capable of producing mixed reality experiences. This award supports fundamental research to endow soft robots with the capability to generate mixed reality experiences. These mixed reality experiences could include highly accurate surgical training simulators as well as educational demonstrations. This research has broad impacts in healthcare, manufacturing, education, and agriculture and will benefit the united states economy and society. This research will provide opportunities for undergraduates at a primarily undergraduate institution to engage in novel robotics research at the intersection of engineering and computer science. Optimization-based techniques enable real-time solutions to the soft robot inverse kinematics problem. However, existing methods have limited application to mixed reality because they only consider physical robots in isolation and cannot coordinate movements with virtual content or maintain physical-virtual coherence during user interaction. This research addresses this GAP by formalizing the simultaneous control of physical soft robots and corresponding virtual avatars and thereby endowing soft robots with the ability to generate mixed reality experiences. The research team will develop a unified model-based optimization framework for controlling mixed reality soft robots, including a novel user-aware immersion GAP constraint, will develop a computational design algorithm for mixed reality soft robots, and will validate their methods using physical soft robots and VR headsets. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are not planned for this award.