Project Grant 2212301

This National Science Foundation Project Grant of $628,966 will fund research from August 2022 to July 2025 to develop technologies for recognizing American Sign Language (ASL) from continuous signing videos. The award was made under the Computer and Information Science and Engineering program (CFDA 47.070), which supports research and education in computing and information sciences.

The grantee, Rutgers University, and its partners will create a novel machine learning approach with three key components: segmentation of regions within continuous signing videos containing distinct sign types; segmentation of individual signs within those regions; and recognition of segmented signs using linguistic constraints and consideration of coarticulation effects. This end-to-end system will explicitly detect sign types, segment signs, and apply type-specific recognition strategies. The research aims to advance sign recognition capabilities from isolated signs to continuous signing at scale, with potential applications in ASL translation, educational tools for learners, and sign search capabilities. All data and methods developed will be publicly shared to facilitate further research.