Representation and computation of semantic categories in the dorsal visual stream

The National Eye Institute awarded The Trustees of Columbia University in the City of New York $225,564 on May 1, 2026, under the Vision Research program (CFDA 93.867) to support analysis and modeling of neurophysiology data from the dorsal visual stream in macaque monkeys trained to categorize visual motion by direction. The project applies representational geometry analysis and artificial neural network modeling to investigate how the primate brain generates neural representations that reflect learned semantic categories based on motion. The work combines population-level analysis of a first-of-its-kind neurophysiology dataset across multiple macaque subjects with computational modeling to test central hypotheses about hierarchical transformation of sensory feature representations through the dorsal visual stream, progressively distorting features to emphasize their learned behavioral meaning. Performance occurs in New York, New York, with period of performance from May 1, 2026, through September 29, 2028.

Abstract

Humans and other primates display tremendous visual intelligence, effortlessly rendering information from an extremely high-bandwidth input modality in a format that is understandable and generally useful for behavior. This occurs through at least two forms of discretization: we identify discrete objects and group them into discrete categories. Such categories are closely related to abstract symbols, neuronal representations of which have been proposed to enable US to form new concepts from familiar ones. This ability is critical for intelligent behavior, highlighting the importance of understanding the symbolic representations that enable IT. While neuronal representations in the late stages of the primate ventral visual stream capture information about 'natural' categories (e.g. shapes, colors, or object types, like cars VS. Faces) through a property called 'disentanglement', the neural basis for decomposition into semantic categories, which are defined through experience, remains unknown. In this project i address this GAP through analysis and modeling of a first-of-its-kind neurophysiology dataset collected across the dorsal visual stream in many macaque monkeys trained to categorize visual motion by its direction. Through a combination of (a) modern population- level analysis of these data leveraging the emerging toolkit of representational geometry, and (b) artificial neural network modeling, this project will address how the primate brain solves the problem of generating representations that reflect semantic categories. Central hypotheses: representations that capture semantic categories based on motion are generated through hierarchically through the dorsal visual stream through a series of transformations that progressively distort sensory feature representations to emphasize their learned behavioral meaning (category) (H1). This enables the behavioral benefits of categorization (interpolation to new stimuli within the same category, more reliable responses) without compromising behaviors depending on veridical representation of stimulus features (H2). Aim 1: i will compare population codes for visual motion direction across representational stages MT-MST-LIP using large-scale macaque neurophysiology data. Aim 2: i will identify computations underlying category-aligned representational changes between stages of the dorsal visual stream using neurophysiology data and artificial neural networks (anns). Aim 3: i will model the behavioral implications of semantic category-aligned representations using multi-layer anns to understand the computational tradeoffs imposed by explicit category representation.