This $499,999 federal Project Grant award from the National Science Foundation's (NSF) Computer and Information Science and Engineering (CISE) program (CFDA 47.070) will support collaborative research to investigate the theoretical foundations of compositional learning in large language models (LLMs) based on transformer architectures.
The research aims to advance the understanding of how LLMs, such as GPT-4, LLAMA 2, and CLAUDE 3, can decompose complex tasks into simpler intermediate steps to tackle text/code generation, arithmetic, problem-solving, and question-answering. The three key research thrusts will explore the expressive capacity of transformers, the statistical properties of compositional learning, and the optimization principles for more efficient training of transformers. The findings will be incorporated into educational curricula and shared through workshops and outreach activities to promote responsible AI practices and AI education. No subrecipient subawards are planned under this award, which has a performance period from July 1, 2024 to June 30, 2028.
Generated 1/28/25, 10:30 AM