Sbir phase i: RE-envisioning ALT text for education through concurrent authoring and diagram design

The National Science Foundation awarded a $275,000 Project Grant to Alchemie Solutions, Inc. under the NSF Technology, Innovation, and Partnerships program (CFDA 47.084) to develop a system for generating alternative text descriptions of science, technology, engineering, and mathematics (STEM) diagrams. The system will produce machine-readable configurations of diagram components and spatial relationships, and generate layered, text-based descriptions matched to a context-driven lexicon. The project aims to go beyond current standards by creating an innovative approach to representing STEM diagrams through alternative text, allowing screen reader users to interact with content tailored to their skills and knowledge. Key objectives include designing a system that automatically produces diagram descriptions as educational resources are constructed and devising a non-linear method for standardized representational layers within the descriptions. The project seeks to advance STEM education accessibility and opportunities for students who are blind or have low vision.

Abstract

The broader impact/commercial potential of this small business innovation research (sbir) phase i project is to create a pathway to success in science, technology, engineering and mathematics (stem) for all students, including those that are blind or have low vision (BLV), with a usable and effective description system for stem diagrams. Only 3% of BLV individuals have been employed in a stem field. The lack of accessible accommodations creates roadblocks for these students in stem education throughout their academic journey, leading many students to give up, even though they have as much potential for academic success as the general population. All web-based, non-decorative diagrams are required to include alternative (ALT) text descriptions for use with screen readers. Current methods for making stem visualizations accessible through ALT text are not scalable or amenable for use with dynamic digital media, nor is ALT text standardized or proven usable by learners, resulting in lost learning opportunities for BLV students. The absence of reliable auto ALT text generation is a roadblock for educational institutions and academic publishers to adopt new instructional technologies, as digital content and learning systems must include accommodations for accessibility. This small business innovation research phase i project will create a first-of-its kind system that generates descriptions of stem diagrams as the educational resources are constructed. This project will go beyond current standards for accommodation, by creating an innovative layered approach to the ALT text representation of stem diagrams so that screen reader users can interact with the content in a manner that is best supported by their skills and prior knowledge. The technical objectives are to: 1) create a system that produces machine-readable configurations for the components of a diagram and spatial relationships between the components, and then generates a text-based description of the diagram and 2) devise a non-linear method for representing stem content through a standardized system of representational layers within the description. Both the general description (objective 1) and layered description (objective 2) will be constructed by matching the diagram configuration to a context-driven lexicon. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.