Benchmark large language models (LLMS) as synthetic survey respondents against real human data from a discrete choice experiment, evaluating whether LLM-generated responses can reliably replicate…

Active Cooperative Agreement Awarded

Award ID
70NANB26H326
Recipient
Awarding agency
Obligated
$95,035

Award facts

Assistance listing
11.068 Advanced Manufacturing
Completion date 15% complete
Place of performance
Washington, DC 20052, USA

Award details come from USAspending.gov.

Description

Purpose
The purpose of this project is to benchmark large language models (LLMS) as synthetic survey respondents against real human data from a discrete choice experiment, evaluating whether LLM-generated responses can reliably replicate structural preference parameters, willingness-to-pay distributions, and response heterogeneity.
Activities
This project will develop a novel empirical framework to evaluate synthetic consumer responses against real human data in a rigorous econometric setting. We will condition multiple llms on detailed demographic profiles and attitudinal characteristics and compare their generated responses to those from an actual survey (on the adoption of used vehicles with different characteristics). The survey integrates both revealed and stated preference questions, including dces designed to estimate heterogeneous WTP. Rather than comparing responses question-by-question in isolation, we will assess both marginal similarities and cross-question internal consistency, particularly within DCE tasks where utility maximization assumptions impose structural coherence. Crucially, we will estimate advanced econometric models, including mixed multinomial logit (MMNL) and latent class (LC) models, to evaluate whether synthetic data can recover systematic heterogeneity in preferences and WTP. By comparing parameter distributions, covariance structures, and segment classifications between synthetic and human samples, we aim to assess not merely surface similarity but structural equivalence.
Expected outcomes
The findings will inform best practices for survey design, detection of synthetic respondents, and the responsible use of llms in data augmentation and methodological research. The findings will produce validated measurement standards, bias diagnostics, and actionable methodological guidelines for the responsible use of synthetic consumers in economic research, directly supporting NIST?S AEO mission to advance measurement science in the public and private sectors with credible, empirically grounded microeconomic analysis tools.
Intended beneficiaries
Survey researchers will benefit from guidance on whether and how llms can be used in survey research for study consumer preference and willingness to pay as well as best practices developed for survey design, detection of synthetic respondents, responsible use of llms in data augmentation, and limitations awareness for early-stage market exploration. Survey data users will benefit from the resulting improvements in data quality resulting from surveys that implement the knowledge gained from this research. The research will also benefit NIST because the research targets a measurement science need that could significantly impact multiple NIST research programs.
Subrecipient activities
The recipient does not intend to subaward funds.

Quick answers

Who received award 70NANB26H326?

70NANB26H326 was awarded to George Washington University (The) by National Institute of Standards and Technology (Department of Commerce) on September 11, 2026.

How much is award 70NANB26H326 worth?

$95,035 has been obligated on 70NANB26H326.

On GovTribe

See this award's full record

  • Every transaction and subaward
  • The recipient's other awards and teaming partners
  • Similar grant awards
  • Ask GovTribe AI about this award