Crew extravehicular activity (EVA) is limited on spaceflight missions. Multiple, small robotic spacecrafts with varying levels of autonomy are needed to perform tasks that might have been completed by…

The Regents of the University of Michigan was awarded a $967,708.00 grant by the National Aeronautics and Space Administration (NASA) to investigate the use of augmented reality (AR) multi-modal interface displays and communication pathways for improving human-robot communication, situation awareness, trust, and task performance during long-duration exploration missions (LDEM). The objective is to develop a simulation and hardware testbed to evaluate these capabilities, which could enable crews on space missions to more effectively process information and communicate with autonomous robots to safely and efficiently complete exterior inspection and repair tasks. This grant does not have a set-aside designation.

Crew extravehicular activity (EVA) is limited on spaceflight missions. Multiple, small robotic spacecrafts with varying levels of autonomy are needed to perform tasks that might have been completed by an astronaut (e.g., an exterior surface inspection or repair). Crews on long duration exploration missions (ldem) will have less access to ground support during task operations. As a result, they will need to process more information and communicate with autonomous robots effectively to ensure tasks are progressing safely and on schedule. The objective of these studies are to investigate the use of augmented reality (AR) multimodal interface displays and communication pathways for improving human-robot communication, situation awareness (SA), trust, and task performance. This will lead to developing guidelines for designing human-robot system interactions that enable operational performance for crews on spaceflight missions. The specific aims are to: 1) develop a simulation testbed for examining communication between human-robot teams. 2) develop a hardware testbed for examining communication between human-robot teams. 3) evaluate human SA, trust, and task performance within a short duration and long-duration ground-based study (simulation and/or hardware) through testing various interface communication modalities and information displays. 4) (option) perform additional studies for alternate parameters of interest that could be tested using the study testbeds. Additional parameters include timing and persistence of information, gesture command mapping, varying the levels of robot automation, evaluating precision enabled by each command mode. 2 the proposed ground testbed will simulate an ldem inspection task with human subjects maneuvering free-flying semi-autonomous satellites to obtain video imagery of the space craft exterior. The satellites used in the simulation study are modeled after the synchronized position hold engage reorient experimental satellites (spheres). Spheres are small free-floating autonomous satellites that have been used to study precision navigation and maneuvering in zero gravity experiments aboard the ISS as well as in ground-based experiments. These satellites are predecessors to the newer astrobees satellites, which are now on the ISS. For the ground-based hardware testbed, we integrate small drones to simulate the 3D positioning of a small satellite for a ground-environment. Video imagery captured by cameras on the drones will be used for automatic scene reconstruction to enhance astronaut perception of the environment. This imaging enables creation of a 3D model of the environment which can then by manipulated in the AR display for navigation and to obtain feedback of the surroundings. For the simulation studies, subjects will be asked to navigate one or more semi-autonomous spheres to perform exterior inspection of a simulated station. Simulations will include surface anomalies that are critical and non-critical that must be discovered subjects will be able to control the direction of the spheres by using keypad-like virtual inputs, gestures, joysticks, and/or by selecting waypoints along a desired pathway.