Sttr phase I: camera-based multimodal AI for health monitoring

Foresightcares Inc. received a $275,992 Project Grant award from the National Science Foundation under the NSF Technology, Innovation, and Partnerships program to develop a multimodal artificial intelligence system for remote health monitoring of seniors. The award period is from March 15, 2022 through February 28, 2023.

Through computer vision, sensor fusion, and artificial intelligence algorithms, Foresightcares will create a training framework to analyze seniors' daily activities, behaviors, and interactions. This will enable holistic and continuous monitoring for early detection of potential health issues like loneliness, behavioral disorders, or confusion. The integrated edge-based Internet of Things solution is designed to provide real-time feedback to caregivers while protecting personally identifiable information. The system offers novel remote health monitoring capabilities with privacy protections for seniors.

Abstract

The broader impact of this small business technology transfer (STTR) phase I project will be to mitigate healthcare costs and challenges associated with the nation's growing aging population. The outcome of this project will be a system that enables holistic, accurate, and faster monitoring of elderly people that will 1) improve quality of life and health of senior citizens, 2) address the future nursing shortage, particularly in the geriatric field, and 3) reduce healthcare costs. The proposed technology can impact a wide range of public health applications that require continuous, non-intrusive health monitoring. This small business technology transfer (STTR) phase I project applies real-time computer vision and sensor fusion processing for real-time continuous monitoring and analysis of health-related activities. In addition, this project takes an essential step beyond current solutions by analyzing seniors' group behavior and interactions with objects to determine whether they suffer from loneliness, behavioral disorders, or confusion states. This project utilizes multiple artificial intelligence algorithms to maximize the discriminative power to understand human micro-body motions and thereby create a training framework for personalized behavioral understanding and monitoring. IT also proposes an innovative solution to leverage the results of computer vision as attention feedback to thermal sensors to enhance the accuracy and robustness of thermal processing against temperature variations. Furthermore, the project offers an integrative edge-based internet of things solution to protect citizens' privacy while providing real-time continuous feedback to responsible personnel. The proposed system offers a novel user experience while protecting personally identifiable information. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.