Project Grant 2551476

The National Science Foundation Division of Computing and Communication Foundations awarded the University of Illinois $999,999 on August 1, 2026, under the Computer and Information Science and Engineering program (CFDA 47.070) to develop novel techniques enabling large language models to succeed at complex mathematical and coding tasks while addressing inference efficiency concerns.

The project focuses on diffusion-based LLMs that achieve global token awareness by modeling joint token distributions. Classical LLMs predict text sequentially, one token at a time, and cannot dynamically correct previously generated tokens if errors occur; this necessitates external error correction or regeneration, increasing computing cost. The project will study two complementary techniques to enable models to capture joint dependencies among concurrently predicted tokens: introduction of a latent variable to anchor sequence intent, and employment of continuous relaxation to transition to a continuous state space. These innovations resolve inconsistencies during prediction rather than requiring post-hoc fixes and address the statistical dependence problem that arises when multiple tokens are jointly revealed in a single time step during faster inference in discrete diffusion formulations.

Work is performed in Urbana, Illinois, with a period of performance from August 1, 2026, through July 31, 2030.