Building safety guards into llms for trustworthy automatic simplification of medical documents
Active Project Grant Awarded Last action
- Award ID
- R01LM014600
- Recipient
-
University Of Texas At Austin UEI V6AFQPN18437 Part of University Of Texas System
- Awarding agency
-
National Institute of Environmental Health Sciences
National Institutes of Health, Department of Health and Human Services
- Obligated
- $1,400,945
Award facts
- Assistance listing
- 93.879 Medical Library Assistance
- Completion date 54% complete
- Place of performance
- Austin, TX 78759, USA
Award details come from USAspending.gov.
Description
- Abstract
- Texts describing medical advances are of keen interest to the general public. However, reliable medical evidence is largely disseminated in peer-reviewed journal articles describing new findings. Because such articles are wrięen in technical language intended for experts, this "primary literature" is effectively inaccessible to the general population. With very large language models (LLMS) like chatgpt now widely available, lay people are increasingly turning to them for medical information. One POTENTIALLY promising use is for LLMS to provide simplified versions of medical papers. However, while LLMS can capably simplify texts automatically, they can also still generate inaccurate, unsupported, and/or POTENTIALLY misleading information, posing a risk. This proposal seeks to develop novel natural language processing (NLP) technologies to mitigate risks and improve the reliability of LLM outputs for the task of medical text simplification. Given the high-stakes of healthcare information, we focus on building controllable, transparent LLMS that are moderately sized, and design tools that enable communities to strike a balance between using LLMS safely and perceiving their outputs critically, while (POTENTIALLY) improving health literacy by eventually empowering the public with more reliable access to high-quality, newly published medical findings. We propose several methodological safeguards. To begin, we will design the first error detection model for LLM-generated simplifications of medical texts, trained with expert-annotated data focusing on factual correctness. This tool will then allow US to build safer knowledge distillation methods, i.e., training much more efficient, smaller models on examples elicited from massive, closed models like GPT-4 calibrated by estimated confidence of their correctness. With full access to the parameters of the distilled model, we propose innovative ways to improve the factuality and readability of the output, and to estimate the model's (UN)certainty of its own output. We will then integrate these safety guards into a prototype, such that they can be evaluated by medical experts and lay readers.
Transactions
| Action | Obligated | Date |
|---|---|---|
| Amendment | $336,527 | |
| Amendment | $336,838 | |
| Amendment | $0 | |
| Amendment | $0 | |
| Amendment | $363,790 | |
| Original award | $363,790 |
Subawards
| Subrecipient | Amount | Awarded |
|---|---|---|
| Georgia Institute Of Technology | $204,544 |
Quick answers
Who received award R01LM014600?
R01LM014600 was awarded to University Of Texas At Austin by National Institute of Environmental Health Sciences (National Institutes of Health) on September 18, 2024.
How much is award R01LM014600 worth?
$1,400,945 has been obligated on R01LM014600.
On GovTribe
See this award's full record
- Every transaction and subaward
- The recipient's other awards and teaming partners
- Similar grant awards
- Ask GovTribe AI about this award