Federal agency name: defense advanced research projects agency (DARPA), defense sciences office (DSO) funding opportunity title: young faculty award (YFA) funding opportunity number:…

The Defense Advanced Research Projects Agency awarded the University of Illinois a $499,999 cooperative agreement on September 30, 2025, under the Research and Technology Development program (CFDA 12.910, Young Faculty Award track).

The University of Illinois will develop an agentic framework that revolutionizes multimodal video understanding by enabling processing and analysis of extremely long video contexts—such as hours-long videos—with multi-hop reasoning across multiple interaction rounds. Unlike existing large multimodal models constrained to single images or short clips, the framework will establish a unified foundation for addressing emerging vision-language tasks including open-world perception, grounding, and multimodal reasoning and question answering. This work supports the advancement of autonomous agents and assistants across virtual and physical domains.

Performance runs through September 29, 2027, with work conducted in Illinois. The cooperative agreement structure contemplates active collaboration between DARPA and the recipient throughout the period of performance.

Federal agency name: defense advanced research projects agency (DARPA), defense sciences office (DSO) funding opportunity title: young faculty award (YFA) funding opportunity number: DARPARA2501CATALOG of federal domestic assistance (cfda) number(s): 12.910 research and technology developmentprogram description: DARPA is soliciting innovative research proposals in the areas of interest to the following DARPA technical offices: defense sciences office (DSO), information innovation office (I2O), microsystems technology office (MTO), and the tactical technology office (TTO). Further detail regarding the specific technical areas of interest can be found below under open office topics. Proposed research should investigate innovative approaches that enable revolutionary advances in science, devices, or systems. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice. We aim to revolutionize multimodal video understanding by developing an agentic framework that interacts with video content over multiple rounds and performs multi-HOP reasoning. Unlike existing large multimodal models, which are constrained to single images or short video clips, our framework enables the processing and analysis of extremely long contexts, such as hours-long videos, with unprecedented depth and efficiency. By scaling the comprehension of complex, long visual sequences, this work will establish a unified foundation for addressing a broad range of emerging vision-language tasks, including open-world perception, grounding, and multimodal reasoning and question answering. As video understanding is central to computer vision, our work is crucial for advancing multimodal systems that will underpin future autonomous agents and assistants across virtual and physical domains. The proposed work is ambitious in the context of current methods, driven by the fundamentalchallenges of processing extremely long token sequences in video understanding. With the rise of large language models (llms) and vision-language models (vlms), data are uniformly represented as sequences of basic units (e.g., words or image patches), known as tokens . However, long videos spanning hours can generate millions of tokens, far exceeding the maximum context length manageable by even the most advanced llms and vlms. For example, the state-of-theart VLM, longvila, can process at most 2,000 frames a sparse sampling rate of less than one frame per second making IT insufficient for capturing fine-grained details, complex temporal dependencies, and the nuanced intentions of users in extended videos.to address these challenges, our key novelty is to introduce a VLM-powered agent with advancedreasoning capabilities that coordinates multi-round video perception, breaking down longvideoreasoning into manageable context lengths. The agent actively interacts with the video overmultiple rounds, constructing an internal world model while selectively attending to relevant segmentsand iteratively refining its understanding much like how a human watches and analyzesdifferent parts of a video. This multi-round perception process overcomes the context length limitationsof existing vlms, enabling more detailed and reliable understanding. Moreover, multi-hopreasoning through speculation and self-reflection resolves ambiguities in user intentions, whichoften depend on a deep understanding of the video context.