Hnds-R: a longitudinal, physician-level dataset from medical directories

This is a $496,927 Project Grant awarded by the National Science Foundation's (NSF) Social, Behavioral, and Economic Sciences (CFDA 47.075) program. The grant was awarded to Trustees of Tufts College on June 1, 2025 and has a planned completion date of May 31, 2028.

The grant project aims to create a novel, publicly-accessible, longitudinal database of physician-level data extracted from American Medical Directories (AMDs) published between 1906-1938. This database will capture detailed information about the early 20th century medical workforce, including physicians' names, practice locations, training histories, specializations, demographics, and professional affiliations. The project will also link these historical AMD records to other relevant data sources to enable in-depth sociological research on the organizational dynamics of the medical profession during this critical period, which saw significant disruptions like the Flexner Report, Great Migration, and World War I. Public-facing outputs and activities will also be developed to bring the utility of this dataset to a broader audience.

Abstract

This project makes american medical directories (AMDS) data accessible to social science researchers by extracting, formatting, and geolocating data from these sources. IT demonstrates the utility of these data by exploring a set of sociological hypotheses, and sustainably archives them for future generations. The AMDS, periodically published by the american medical association from 1906, present an immense opportunity to illuminate the organizational dynamics of professional medicine in the early 20TH century. In addition to physicians? Names and practice locations, these volumes also contain valuable information about individuals? Training histories and medical specializations, demographic characteristics, and membership in state and local societies. Because AMDS were published triennially, they also present an opportunity to link individuals over time, exploring physicians? Movement between regions, as well as how and where training pipelines for the medical workforce developed. No other data source offers such nuanced, individual-level information about the early medical workforce, yet the AMDS remain underexplored archival sources, largely due to the difficulties of extracting large quantities of data from original archival sources. Additionally, public-facing outputs and activities of the project bring its utility to a broader audience, enabling still further kinds of non-academic inquiries and applications. This project centers on a critical period of history in professional medicine and captures significant demographic and cultural moments that likely interrupt previous patterns, including the flexner report, great migration, and world war I. Specifically, the project: 1) delivers a novel, publicly-accessible, longitudinal physician database, extracted from purposively selected editions of the AMD between 1906 and 1938; 2) links these records to other relevant historical data sources, including decennial censuses, leveraging existing NSF-funded data infrastructures; 3) tests specific sociological hypotheses related to practice markets, growth patterns, physician migration, and specialization to demonstrate the dataset?S utility; and 4) engages broader audiences through data visualization, media, and public events. The project enables interdisciplinary quantitative and mixed methods analyses that previously were limited by the inaccessibility of AMD records, and provides a ?gold standard? Version of extracted records that could be used in future efforts to extract data from additional AMD editions. By building a novel, longitudinal dataset of the medical workforce between 1906-1938, the project provides a through-line from the early days of professional medicine in the US, and, critically, makes a trove of physician-level data accessible to social science researchers and the broader public. This project is jointly funded by the sociology program and the human networks and data science-research (HNDS-R) program. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are planned for this award.