This National Science Foundation (NSF) Technology, Innovation, and Partnerships (CFDA 47.084) SBIR Phase I award of $275,000 to Relai, Inc. will support the development of innovative methodologies to enhance the reliability of large language models (LLMs). The key products to be delivered include:
-
Methodologies to inspect and mitigate jailbreaking issues in LLMs, where adversarial prompts can circumvent their alignment. This work aims to identify vulnerabilities in current LLMs and propose robust countermeasures to fortify these models against sophisticated attacks.
-
Methodologies to inspect and mitigate LLM hallucinations, where models can generate non-factual responses. This involves sophisticated analysis of model outputs to provide deeper insights into the internal workings of LLMs.
-
Methodologies to inspect and mitigate biases in LLMs, which is critical for ensuring ethical and fair artificial intelligence (AI) applications.
The project will integrate these advanced tools into a comprehensive, user-friendly, and unified platform to establish a new benchmark for the development and deployment of reliable AI applications. The award period is from Sep 1, 2024 to Aug 31, 2025.
Generated 3/4/25, 8:08 AM