Gaze-contingent AI-enabled low vision assistive technology (GALVAT) - smart assistive technology (SAT) for individuals with severe vision loss is becoming affordable, thanks to miniaturization of…

Johns Hopkins University received a $621,250 Project Grant award from the National Eye Institute (NEI) under the Vision Research program (CFDA 93.867) on September 15, 2025, with a completion date of August 31, 2027. The project, titled "Gaze-Contingent AI-Enabled Low Vision Assistive Technology (GALVAT)," will develop smart assistive technology designed to address information overload challenges in vision loss rehabilitation devices. The research consortium—comprising Johns Hopkins University, Chicago Lighthouse, Applied Universal Dynamics Corp (AUD), and the Pritzker Institute at the Illinois Institute of Technology—will design and prototype gaze-contingent systems that reduce information overwhelm by defining a region of interest specific to each user's gaze and remaining visual field. The R61 phase will utilize a head-mounted display with built-in eye-tracking hardware and encompass user requirement formulation through interviews and expert user panel recruitment.

The project outputs include formulated user requirements for smart assistive technology users, HMD-based prototype development incorporating gaze-contingent artificial intelligence capabilities, and design specifications informed by the lived experience of individuals with severe vision loss. Applied Universal Dynamics Corp serves as a sub-awardee, contributing expertise in smart systems design and integration (awarded December 10, 2025). The clinical and rehabilitative knowledge provided by Johns Hopkins' Low Vision Division and Chicago Lighthouse will inform ergonomic design ensuring non-visual device access, while future R33 phase work will transition the technology to a glasses-frame platform with smartphone or belt-pack computing to expand accessibility to visual prosthesis users.

Gaze-contingent AI-enabled low vision assistive technology (GALVAT) - smart assistive technology (SAT) for individuals with severe vision loss is becoming affordable, thanks to miniaturization of display hardware (head-mounted or phone screen) and more powerful AI software, driven by demand in the consumer market. Unfortunately, these developments do not take into account visually impaired users' perspective on: 1) ergonomics, i.e., non-visual access to all device functions; and 2) relevance, i.e., functionality that meets their needs and enhances independence. The most often heard complaint is that the device or APP overwhelms the user with information, so pertinent information is too hard to find. In this application, four organizations - the low vision divisions at johns hopkins (JHU) and the chicago lighthouse (CLH), applied universal dynamics corp (AUD) and the pritzker institute at the illinois institute of technology (IIT) - are joining forces to address this problem. We are uniquely positioned to do so, through our combined expertise in smart systems design and integration (AUD), clinical and rehabilitative knowledge of the user population (JHU and CLH), and the opportunity to extend the user base to visual prosthesis users (IIT). With the users' lived experience in mind, we propose to reduce the information overload by defining a gaze-contingent region of interest (ROI) specific to each user's gaze and remaining visual field, so the AI- supported system will operate on targets within this ROI. In the R61 phase of the project, we will use a head- mounted display (HMD) with built-in eyetracking as the platform; in the R33 phase, a glasses frame with built-in camera and eyetracking will be used, with a smart phone or belt pack computer running the AI engine. The R61 phase will encompass 3 aims: 1) formulate user requirements by surveying SAT users (interviews and discussions), and recruit an expert user panel for ongoing feedback; 2) design and build an HMD-based prototype (H1) for clinical tests, with a software upgrade (H2) based on initial outcomes and feedback; 3) conduct user training and collect ongoing feedback and performance data for evaluation and RE- design. The R33 phase will create the glasses-based system and enhance processing power and customization, through 3 additional aims: 4) integrate the glasses frame camera and eyetracker with a speech- enabled belt pack computer (G1), with an upgrade in year 4 (G2) suitable for take-home use; 5) enhance user- specific solutions by adding individualized and task-specific ROI adjustments, and add search and hazard warning functions for users with narrow fields; 6) conduct user training and collect ongoing feedback and performance data for evaluation and G2 upgrades, followed by a 3-month take-home study with performance tests (pre VS, post). The resulting gaze-contingent AI-supported low vision adaptive technology (GALVAT) will be highly adaptive to users' individual needs and greatly contribute to their functional visual independence.