This $300,000 Project Grant award from the National Science Foundation's (NSF) Computer and Information Science and Engineering (CISE) program supports research to improve the evaluation and functionality of large language models (LLMs). The University of Texas at Austin will lead this 3-year collaborative research project to identify a taxonomy of "evaluation concepts" for assessing LLM responses and develop technology to automatically evaluate and improve LLM performance based on...
This $875,000 Project Grant was awarded by the National Science Foundation (NSF) under the Computer and Information Science and Engineering (CISE) Federal Grant Program (CFDA 47.070) to the University of California, San Diego (UCSD). The project aims to develop new techniques for aligning large language models (LLMs) with formal specifications in order to generate high-quality computer code that provably matches user intent. Specifically, the project will: (1) develop grammar-aligned decoding...
This Project Grant award of $500,000 from the National Science Foundation's (NSF) Computer and Information Science and Engineering (CISE) Federal Grant Program supports research to understand and mitigate security vulnerabilities in machine learning (ML) models. The research aims to characterize how malicious actors could exploit the unused parameters in trained ML models to install covert functionality, and develop mitigation approaches to improve the robustness and trustworthiness of ML...
This $100,000 Project Grant was awarded by the National Science Foundation's (NSF) Computer and Information Science and Engineering (CISE) program (CFDA 47.070). The grant supports the development of an open, community-driven evaluation infrastructure to assess the safety risks of large language models (LLMs). The project aims to: (i) conduct surveys and interviews with experts to identify critical safety concerns and evaluation gaps; (ii) organize a workshop to refine the evaluation...
This Project Grant award of $675,000 was provided by the National Science Foundation (NSF) under the Computer and Information Science and Engineering (CFDA 47.070) program. The award aims to improve the ability of large language models (LLMs) to generate high-quality source code by deeply integrating program analysis techniques into the LLM training, code generation, and evaluation processes. The key products and services to be delivered include: Developing novel quantitative program analysis...
This Project Grant award, provided by the National Science Foundation (NSF) under the Computer and Information Science and Engineering (CFDA 47.070) program, aims to understand the effects of large language models (LLMs) on the work of online information professionals. The $370,692 award to the University of Washington will fund research to develop an epistemological framework for characterizing the risks posed by LLM-generated content, as well as proactive and reactive approaches to assessing...
The National Science Foundation (NSF) awarded a $800,000 Project Grant under the Computer and Information Science and Engineering (CFDA 47.070) program to the University of Illinois Urbana-Champaign. The 3-year grant, effective September 1, 2024, focuses on enhancing the safety of large language models (LLMs) used in high-stakes applications. The project aims to develop quantifiable safety measures and algorithms to detect and mitigate unsafe behaviors in LLMs, such as providing false or...
This $100,000 Project Grant award from the National Science Foundation (NSF) Computer and Information Science and Engineering (CFDA 47.070) program aims to develop a holistic benchmarking infrastructure for evaluating large language models used in software engineering. The key activities include: Conducting surveys and interviews with the software engineering and machine learning research communities to gather requirements and understand barriers in evaluating large language models for code....
This three-year $800,000 project grant from the National Science Foundation's Computer and Information Science and Engineering program aims to advance understanding of large language models through mathematical and conceptual analysis. The Trustees of Princeton University will receive funding to develop simplified generative text models, analyze how language models are trained on such generated texts, examine why learned models can perform downstream tasks, and design new adaptation methods with...
This $500,000 project grant from the National Science Foundation's Computer and Information Science and Engineering program aims to develop new techniques for software vulnerability discovery. Specifically, the University of California, Riverside will receive funding from February 2022 through January 2025 to create a fast binary code concolic execution engine and dual concolic execution approach that combines source code and binary code analysis. These new methods seek to significantly...