Collaborative research: estimating articulatory constriction place and timing from speech acoustics

Ended Project Grant Awarded Last action

Award ID
2141275
Recipient
Awarding agency
Obligated
$17,259

Award facts

Assistance listing
47.075 Social, Behavioral, and Economic Sciences
Completion date
Place of performance
New Haven, CT 06511, USA

Award details come from USAspending.gov.

Description

As the agency reported it.

Abstract
This collaborative project focuses on a new approach for using speech recordings to study speaker pronunciation habits – that is, the way speakers systematically coordinate the articulatory movements of their lips, jaw, tongue, GLOTTIS and soft palate to produce words and sentences. These articulatory habits differ between individuals, and across languages and dialects of the same language, accounting for many aspects of foreign accent, speech disorders and speaking style. Whereas previous studies of these habits have required specialized equipment for the immediate observation of articulator movements, the aim of this project is to develop and improve a tool for speech inversion – that is, a tool that can accurately recover articulatory movements directly from the acoustic speech signal using machine learning methods. To date, the tool developed by the project team has successfully recovered movements of the tongue and lips; the current project extends the tool?S functionality to encompass nasality (soft palate) and voicing (GLOTTIS). Training and validation of the extended system will proceed using a newly collected corpus of acoustic and articulatory data drawn from speakers of american english. This corpus, comprising CO-collected audio, nasal, voicing, and articulatory movement, will serve as 'ground truth' for training and assessing the capabilities of the fully trained speech inversion system. As a further test, we will test IT against ground truth data from speakers of languages with patterns of articulatory habits known to differ from english. The goal of this project is to develop and refine a speech inversion tool that 'reads' acoustic recordings of speech and 'recovers' details of the magnitude and timing of articulatory movements. The project aims to accomplish this goal by training specialized neural network models to relate features of the acoustic signal to separately acquired ground-truth nasal VS. Oral outflow signals and concurrent electroglottography. Training data derives from native speakers of english; validation and tests for generalization include productions of speakers of canadian french and russian. When successfully validated, the resulting speech inversion tool will be useful for identifying medical issues that affect speech movement organization, such as the well-known disruption of oral/laryngeal timing in speakers with dysarthria. In addition, incorporating estimates of articulation may also aid in the tracking of changes resulting from medical conditions such as depression and schizophrenia. More generally, the ability to rapidly and easily analyze articulatory movements obtained from audio recordings alone has the potential substantially improve automated speech recognition (ASR) systems, and to assist scholars, forensic scientists, and clinical professionals studying the speech of communities under field conditions in rural or under-resourced areas, and to help in the documentation of endangered languages. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.

Transactions

Action Obligated Date
Amendment -$136,741
Original award $154,000

Quick answers

Who received award 2141275?

2141275 was awarded to Haskins Laboratories, Inc. by Division of Behavioral and Cognitive Sciences (National Science Foundation) on June 15, 2022.

How much is award 2141275 worth?

$17,259 has been obligated on 2141275.

Which agency awarded 2141275?

Division of Behavioral and Cognitive Sciences (National Science Foundation) awarded 2141275, with funding from National Science Foundation.

On GovTribe

See this award's full record

  • Every transaction and subaward
  • The recipient's other awards and teaming partners
  • Similar grant awards
  • Ask GovTribe AI about this award
Try for free Login