Collaborative research: vines: track 1: NSF-SRC: sense-VR: semantic encoding and sensor-driven communication for privacy-preserving multi-site virtual reality systems

The National Science Foundation Division of Information and Intelligent Systems awarded New Jersey Institute of Technology $250,000 on October 1, 2026, under the Computer and Information Science and Engineering program (CFDA 47.070) for collaborative research on semantic encoding and sensor-driven communication for privacy-preserving multi-site virtual reality systems.

The project develops methods to transmit only the most relevant information while protecting user data, enabling virtual reality access in areas with limited connectivity. Work is organized into three interconnected tasks: semantic-aware transmission methods coupled with neural enhancement to recover rich content from limited data; integrated sensing and communication to track user motion and environment changes in real time and adapt communication accordingly; and privacy-preserving computation methods including client-side filtering and secure distributed processing for multi-site virtual reality. System-level designs explore performance, reliability, privacy, and security tradeoffs.

The award funds research at Newark, New Jersey. Performance runs through September 30, 2029. This is an international collaborative project bringing together investigators from Portland State University, New Jersey Institute of Technology, and Linkoping University.

Abstract

Virtual reality can support remote collaboration, education, and healthcare; on the other hand, this often requires fast connections while raising concerns about privacy of user data. These connectivity challenges are especially difficult in rural areas with limited internet access. This project develops new methods to send only the most important information while protecting user data, enabling more people to access immersive and reliable virtual reality experiences. The intellectual merit of this project lies in advancing the foundations of efficient, reliable, and privacy-preserving multi-site immersive communication systems. The work is organized into several interconnected tasks that address communication efficiency, sensing accuracy, and privacy protection. The first task develops semantic-aware methods that identify and transmit only the most relevant information, coupled with neural enhancement methods that recover rich content features from limited received data. The second task focuses on integrated sensing and communication to track user motion and environment changes in real time, adapt communication accordingly, and enhance immersion. The third task designs privacy-preserving computation methods, including client-side filtering and secure distributed processing, in the context of multi-site virtual reality. These tasks are integrated into system-level designs that explore tradeoffs among performance, reliability, privacy, and security. This international collaborative project brings together investigators from portland state university, new jersey institute of technology, and linkoping university to improve shared virtual reality experiences for people in different locations. The broader impacts of this project include expanding access to virtual reality technologies for all communities and helping reduce digital access gaps in education, employment, and healthcare. By enabling efficient operation over limited bandwidth, the project supports wider adoption of immersive tools in schools, remote work, and telemedicine. The project will deliver publicly available tools, datasets, and software modules to accelerate research and innovation, and will integrate its findings into undergraduate and graduate curricula in wireless communication, distributed computing, and immersive media. Collaboration with industry partners will support technology transfer and alignment with real-world needs. The project will engage students at multiple levels through research training, internships, mentoring, and international exchange, and will disseminate results through workshops, open repositories, and publications to ensure broad and lasting impact. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are not planned for this award.