Collaborative research: HCC: computer aided design for nonvisual computing

The National Science Foundation Division of Computing and Communication Foundations awarded Brown University $334,999 on October 1, 2026, under the Computer and Information Science and Engineering program (CFDA 47.070) to develop the first fully accessible 3D computer-aided design environment for blind and low-vision users.

The project will create foundational tactile interaction techniques that enable precise understanding and manipulation of 3D mechanical models on low-resolution displays and dynamic raised-dot tactile arrays. The work integrates AI-driven view optimization, verbal descriptions of model features, and advanced haptic graphics to support professional participation in engineering, architecture, and industrial design. The team will conduct deep co-design collaborations with blind and deafblind leaders to establish methods for precise control over 3D model understanding and design. Development work includes novel techniques for summarizing model features through text, AI-supported navigation recommending optimal viewing angles, and assessment of participant ability to understand models, model edits, and their 3D printed counterparts.

Performance occurs at Brown University's Providence, Rhode Island location, with work extending through September 30, 2029. The assistance type is a project grant.

Abstract

Designing precise 3D mechanical objects is exceptionally difficult on low-resolution displays. This challenge is further compounded when shifting from visual creations to tactile interfaces that utilize dynamic, raised-DOT arrays. While such tactile technology offers significant potential for the people with limited vision (i.e., the blind and low vision (BLV) community), there is a critical lack of standardized methods for viewing 3D models at these restricted resolutions. This project aims to develop the first fully accessible 3D modeling environment for BLV users by integrating AI-driven view optimization and verbal descriptions with advanced haptic graphics. By enabling the precise manipulation of complex models, this work facilitates new opportunities for professional participation in fields such as engineering, architecture, and industrial design. This opens a new arena of work to BLV people, who can now specify and understand complex and precise physical and mechanical models required for full participation in their careers. The goal of this project is to establish methods for precise and expressive control over 3D model understanding and design. The approach that will be in use is to develop deep collaborations within the BLV community, including CO-design with blind and deafblind leaders working towards inclusion in tactile graphics and CAD modeling. First, the project will develop foundational new tactile interaction techniques for precisely understanding 3D models, along with novel techniques for richly summarizing features using textual descriptions. AI will support navigation by recommending optimal viewing angles and summarize changes to a model in text. The study will assess participant ability to understand models and model edits, and to connect them to their 3D printed counterparts. Second, the project will support designing new models through learning about, selecting, and parameterizing features applied to specific edges, vertices, and faces of CAD models. AI support will include language-based search both for features and for applying features to models. Finally, the project will support editing existing models. This will require natural language summaries of geometric changes and debugging that can enhance the model editing experience alongside tactile displays. The project will develop educational materials and test users? Ability to set and achieve editing goals and debug models. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are not planned for this award.