Career: context-aware resource efficient natural-language queries over longitudinal wearable sensor data

The National Science Foundation Division of Information and Intelligent Systems awarded Worcester Polytechnic Institute $396,382 under the Computer and Information Science and Engineering program (CFDA 47.070) on October 1, 2026, to develop a natural-language query system for wearable sensor data that performs computation primarily on edge devices rather than requiring cloud upload.

The project addresses the gap between wearable sensor collection and usable analysis by enabling non-technical users—parents, patients, caregivers, and frontline workers—to ask plain-language questions about longitudinal sensor streams and receive actionable answers. Work organizes into three thrusts: compact multi-resolution storage of sensor histories on single devices; a reasoning layer that weights sensor streams by query context and resolves conflicting signals; and an adaptive runtime that uses early stopping, computation trimming, and selective cloud handoff to manage device resource constraints. The system targets settings with limited computing infrastructure, including home care, rural clinics, and school programs. Real-world usability will be demonstrated using data from an infant behavior study showing how the system helps families understand daily routines.

Performance occurs at Worcester Polytechnic Institute in Worcester, Massachusetts, with a period of performance from October 1, 2026, through September 30, 2031.

Abstract

Wearable devices such as fitness trackers and smartwatches collect large amounts of information about movement, sound, and the body, but the tools that come with them usually show only simple charts that do not answer the questions people actually have. This project builds a system that lets anyone ask everyday questions in plain language, such as how well a baby slept last week or whether a night-shift worker became too warm, and receive clear answers drawn from the sensor information. The system is designed to perform its processing primarily on smartphones, wearables, or other everyday devices, so IT responds quickly and keeps personal information on the device rather than always uploading IT to the cloud. The research is organized into three connected thrusts. The first thrust develops a way to store long histories of sensor data on a single device by saving information in compact form and at several levels of detail, so that both broad trends and brief moments can be recovered when a question calls for them. The second thrust builds a reasoning layer that weighs each sensor stream according to the question being asked and settles cases where two signals point to different conclusions. The third thrust designs a runtime that adjusts how much computation each question requires, using early stopping, trimming of unneeded work, and selective handoff of difficult cases to the cloud when local limits are reached. The work makes information from sensors useful to people who are not technical experts, including parents, patients, caregivers, and frontline workers, and IT supports settings such as home care, rural clinics, and school programs where computing resources are limited. The real-world usability of the system will be demonstrated with data from an infant behavior study, showing how IT helps families understand daily routines. To support reuse and community progress, open datasets, models, evaluation tools, and course materials will be released for public use, and an annual open data challenge will invite undergraduates, high school students, and researchers at under-resourced institutions to take part. Summer workshops for high school students and participation in national research programs for undergraduates and teachers will help strengthen the science and engineering workforce. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are not planned for this award.