Project Grant 2539394

The National Science Foundation Division of Information and Intelligent Systems awarded the New Jersey Institute of Technology $434,844 on August 1, 2026, under the Computer and Information Science and Engineering program (CFDA 47.070) to develop opportunistic and adaptive multi-modal in-sensor computing architectures for artificial intelligence of things applications.

The project advances the design of intelligent sensing systems through three interconnected research thrusts. The first develops reconfigurable architectures and circuits for multi-modal in-sensor accelerators that move computation to sensor nodes efficiently. The second focuses on software-based modality sequencing mechanisms, energy-performance controllers for multi-modal in-sensor accelerators, and graph neural network-based prediction mechanisms to strategize execution paths at the system level. The third thrust integrates these components into a unified framework by developing a large language model-driven accelerator generation platform, supported by structured datasets and automated pipelines. The project establishes a comprehensive evaluation framework, including prototyping and open-source tools, to validate performance across real-world scenarios.

The research enables everyday smart devices such as cameras, health monitors, and environmental sensors to process information directly at data collection points rather than transmitting raw data elsewhere, reducing energy consumption and computational burden. Intended applications include healthcare, safety, and environmental monitoring.

Performance of work occurs in Newark, New Jersey. The project period runs through July 31, 2031.