Understanding equivalent equations: using clinical interviews and nominal response cognitive diagnostic models to measure and describe students? Sensemaking

The National Science Foundation Division of Undergraduate Education awarded $1,105,788 to the Research Foundation of The City University of New York (doing business as Rfcuny - Borough Manhattan Community College) on October 15, 2026, under the STEM Education program (CFDA 47.076).

The project will develop assessment tools to measure student understanding of equivalent equations using cognitive diagnostic models and clinical interview methods. Researchers and instructors will use these assessments to evaluate what students understand about core concepts needed to work with equations. The project employs nominal response cognitive diagnostic models combined with in-depth cognitive interviews to describe new conceptions of equivalent equations not documented in existing literature, define boundaries of existing conceptions, and explore relationships between conceptions, including which commonly co-occur and whether hierarchical relationships exist among them. The mixed-methods approach aims to produce assessments that capture a wider range of student thinking than existing tests and can be validated or adapted to other educational contexts such as high school. The methodology developed for equations assessment has potential application as a model for developing assessments on other topics.

The project responds to the shortage of STEM graduates by addressing algebra as a barrier to STEM degree attainment. Instructors could use the resulting assessments to improve instruction, and researchers could assess the effectiveness of different courses and teaching approaches.

Performance is based in New York, New York, with period of performance from October 15, 2026, through September 30, 2029.

Abstract

This project aims to serve the national interest by developing a tool that researchers and instructors can use to assess what students understand about core ideas needed to work with equations. The U.S. Currently faces a shortage of stem graduates, and algebra is often a barrier to earning stem degrees. This project aims to provide instructors and researchers with assessments to measure student understanding of equations, using innovative statistical techniques to measure a wider range of student thinking than existing tests. Equations are used extensively across stem fields to model real-world problems, and therefore any tools that strengthen students? Understanding of equations have the potential to substantially expand and strengthen the stem workforce. Instructors could use assessments produced by this project to improve their instruction, and researchers could use them to assess the effectiveness of different courses and teaching approaches. Once developed, assessments and learning models from this project will also have the potential to be validated or adapted to other contexts, such as high school. In addition, the innovative methods used by this project to develop assessments could then be used as a model for developing more effective assessments on other topics. This mixed methods project will combine nominal response cognitive diagnostic models (nrcdms) and in-depth cognitive interviews to describe new conceptions of equivalent equations not documented in existing literature, better define the boundaries of conceptions mentioned in existing literature, and explore the relationships between conceptions (e.g., analyzing which conceptions commonly CO-occur and whether some conceptions have a hierarchical relationship with one another). Nrcdms are an innovative approach that draws on the full range of student responses. The most commonly used psychometric models discard information about the specific answer option students chose by scoring each item only as correct/incorrect. In contrast, nrcdms model the specific answer options chosen, allowing for analysis of larger student response patterns that correspond to specific conceptions that each answer option was designed to measure. The goal of this project will be twofold: 1) to generate a robust framework of conceptions that describes the varied ways that college students think about equivalent equations; and 2) to produce a diagnostic instrument that instructors and researchers can use to determine which conceptions of equivalent equations students may hold, which can subsequently be used in future to better adapt and differentiate curriculum and instruction and to inform research. This project thus has the potential to contribute to a richer understanding of how students think about equivalent equations and provide a useful diagnostic instrument for practitioners, as well as to provide an assessment development model that can subsequently be used to develop other diagnostic instruments to measure conceptions in other mathematical domains. This project plans to share assessments, learning frameworks, and methods of assessment development resulting from this research with practitioners, researchers, policymakers, and the general public through a wide variety of dissemination efforts. This project is supported by NSF?s EDU core research (ECR) program and NSF's iuse: EDU program. The iuse: EDU program supports research and development projects to improve the effectiveness of stem education for all students. The ECR program emphasizes fundamental stem education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are planned for this award.