Temporal context in brains and transformers

The National Institutes of Health Office of the Director awarded The Salk Institute For Biological Studies $2.642 million on September 30, 2025, under the Trans-NIH Research Support program (CFDA 93.310) to investigate temporal context mechanisms in long-term working memory by studying cortical traveling waves as a potential biological analog to transformer architectures in large language models.

The research aims to discover how global dynamical activity in the cortex and medium-term synaptic plasticity in cortical circuits create temporal context extending over minutes and hours. The project will employ multichannel recordings to test the hypothesis that cortical traveling waves organize global processing and storage of new memories over minutes and their retrieval and maintenance over hours, operating as a spacetime coding mechanism complementary to rate codes in sensorimotor processing. The investigation includes examination of how cortical neurons integrate direct sensory input from the thalamus with indirect inputs from distant neurons via traveling waves delayed by slow, unmyelinated, long-range horizontal connections.

Performance takes place at The Salk Institute's facility in La Jolla, California, with a period of performance extending from September 30, 2025, through July 31, 2030. The award is structured as a Project Grant, one of several Pioneer Awards offered through the NIH Office of the Director focusing on fundamental neuroscience research.

Abstract

Long-term working memory allows US to remember what was said and seen on the slides at the end of a lecture, what was read in a book after long sequences of saccadic eye movements, and the musical progression during a symphony. The interplay between long-term memory retrieval in many cortical areas is involved. However, the neural mechanisms that allow US to integrate new detailed information and keep IT fresh in mind over minutes and hours are unknown. Brains extract meaning from the temporal context of words in a sequence. This pioneer award aims to discover how global dynamical activity in the cortex and medium-term synaptic plasticity in cortical circuits create temporal context extending over minutes and hours. Transformers in large language models can also keep temporal context over long dialogs and documents by storing them in a long input vector, simultaneously making all past information available. Brains DON't have digital memories like computers, but nature could have evolved alternative mechanisms. Multichannel recordings have revealed traveling waves of neural activity in multiple sensory, motor, and cognitive systems. Traveling waves are ubiquitous in the cortex and carry information from the past to the future. My working hypothesis is that cortical traveling waves may be a dynamical version of the long input vector in transformers. Spacetime coding by cortical traveling waves is complementary to the rate code used for well-studied, fast sensorimotor processing. IT could be used in sensory and memory systems to organize the global processing and storage of new memories over minutes and their retrieval and maintenance over hours. A cortical neuron that receives direct sensory input from the thalamus also receives indirect inputs from distant neurons in the form of a traveling wave delayed by slow, unmyelinated, long-range horizontal connections. Mixing present and past information creates a new type of spacetime population code that simultaneously represents neural activity occurring within a time window and provides temporal context between past and present inputs. The time windows for spacetime codes will vary over the cortex, from 100 MS in V1 to minutes in the prefrontal cortex. However, another mechanism is needed to extend long-term working memory to hours. The frequencies of cortical waves during awake states range from 10 to 40 HZ, which can trigger the suboptimal induction of spike-timing-dependent synaptic plasticity (STDP) lasting 20-60 minutes. This matches the time window for long-term working memory and has the advantage that IT can be reused. Long-term working memory will be simulated and studied in two-dimensional very-large-scale models (VLSMS) of the spiking recurrent cortical networks that support traveling waves and STDP. Experimental collaborators will test predictions from the models in rats, monkeys, and humans. New analytic techniques will be needed to decode the spacetime code from cortical recordings from a wide range of cortical areas.