Collaborative research: icbr: toward efficient statistical computation for high-dimensional data in ecology and evolutionary biology

The National Science Foundation Division of Biological Infrastructure awarded Chatham University $234,805 on August 1, 2026, under the Biological Sciences program (CFDA 47.074) to develop computational methods for statistical analysis of high-dimensional biological data in ecology and evolutionary biology.

The project addresses a core limitation in current statistical practice: existing hypothesis-testing frameworks cannot efficiently process datasets with far more variables than observations (such as genomic, proteomic, or facial recognition data), because computational costs in memory and processing time increase exponentially as data volume grows. The investigators will pursue two research objectives to reduce this computational burden. The first develops a compressed likelihood strategy to calculate model likelihoods when variables exceed the number of research observations, identifying robust lower-dimensional matrix determinants consistent with higher-dimensional estimates. The second objective focuses on developing recursive algorithms to accelerate likelihood computation for high-dimensional linear models. The project will produce open-source software products, deliver mentorship and training opportunities, and organize workshops to disseminate methods to students, researchers, and the broader scientific community. Work is performed at Chatham University in Pittsburgh, Pennsylvania, with performance running through July 31, 2029. The project advances NSF priorities in artificial intelligence.

Abstract

In this project, the investigators will develop new computational methods that will enable proficient analysis of high-dimensional (HD) biological data (data that contain many more variables than observations). Researchers are now capable of collecting large volumes of data, efficiently, because of modern technological advances. These data often comprise hundreds or thousands of variables (e.g., facial recognition data, protein architectural data, or genomic data) per research subject. Although HD data enable extending research frontiers in ecology and evolutionary biology, hypothesis testing frameworks for HD data have not kept pace with the technical advances of data collection. The chief problem is that the computational cost of analysis (computer memory and time) with currently available methods tend to increase exponentially, as data increase. In this project the investigators will address much-needed statistical computation needs for HD data via two research objectives intended to reduce computational burden. This project will promote broader impacts by developing open-source products, fostering human resource development through mentorship and training opportunities, and organizing workshops to disseminate the latest scientific techniques to students, researchers, and the broader community. This project advances NSF?s priorities in artificial intelligence. The investigators will address limitations for calculating linear model likelihoods with HD data. A current challenge is the inability to calculate model likelihoods if variables exceed the number of research observations. A compressed likelihood strategy will be developed that finds robust lower-dimensional matrix determinants that are consistent with higher-dimensional estimates, enabling likelihood estimation for HD data. Second, the investigators will develop recursive algorithmic methods that avoid calculating large, intermediate matrices that are typical in the statistical computations used in current methods. These advances will offer computational solutions that are independent of data size, with low computational cost, and easy parallelization, making linear computation-time analyses attainable. Together, these research objectives will make IT possible for analysis of HD data to keep pace with the technological advances that produce such data. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are not planned for this award.