Evaluation, discovery, and trust in the age of AI-generated content

The National Science Foundation Division of Computer and Network Systems awarded the National Academy of Sciences $200,000 on October 1, 2026, under the Computer and Information Science and Engineering program to produce a publicly available, peer-reviewed issue paper examining how institutions maintain trustworthy and resilient processes for evaluating and navigating information as machine-generated content becomes abundant.

The project convenes experts in artificial intelligence, cybersecurity, information science, research evaluation, and governance to analyze existing evaluation, review, curation, cybersecurity, and trust mechanisms. The analysis characterizes threat models including impersonation, provenance uncertainty, automated spam, and gaming of reputation and review systems, and examines mechanisms for provenance, authentication, identity, and reputation management. The project frames the challenge as one of institutional and cyber resilience rather than simply detecting machine-written text, and compares pressures and workable responses across scientific publishing, government agencies, online platforms, knowledge repositories, and creative marketplaces.

The National Academies Press will publish the individually authored, peer-reviewed issue paper. Place of performance is Washington, DC. The period of performance runs from October 1, 2026, through September 30, 2027.

Abstract

Artificial intelligence tools now draft, revise, translate, summarize, and synthesize content at very low cost. Institutions that review and organize information, including scientific publishers, government agencies, online platforms, knowledge repositories, and creative marketplaces, were built for a world in which producing substantive content required significant human effort. As that effort declines, the volume of material needing evaluation grows, while expert attention and review capacity remain scarce. This project convenes experts and produces a publicly available issue paper on how institutions maintain secure, trustworthy, and resilient processes for producing, evaluating, and navigating information as machine-generated content becomes abundant. The project's novelties are its treatment of this shift as a challenge for institutional and cyber resilience, rather than simply a problem of detecting machine-written text, and its comparison of scientific publishing with other information-intensive settings facing similar pressures. The project's broader significance and importance are that dependable evaluation processes support scientific progress, effective governance, commerce, and cultural production, and that identifying shared pressures and workable responses helps institutions adapt before core review, curation, and accountability processes are overwhelmed. A convening of experts in artificial intelligence, cybersecurity, information science, research evaluation, and governance informs an individually authored, peer-reviewed issue paper published by the national academies press. The analysis assesses existing evaluation, review, curation, cybersecurity, and trust mechanisms; characterizes threat models including impersonation, provenance uncertainty, automated spam, and gaming of reputation and review systems; and examines mechanisms for provenance, authentication, identity, reputation, accountability, and disclosure. IT also considers machine-assisted review, synthesis, and workflow triage. Scientific publishing and research evaluation serve as leading examples, with attention to regulatory comment processes, online platforms, knowledge repositories, and creative ecosystems. Comparing domains surfaces common challenges, transferable approaches, and domain-specific differences. The project clarifies research needs and practical options for sustaining trustworthy information ecosystems and disseminates results to researchers, funders, publishers, institutional leaders, platform operators, policymakers, and technical experts. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the foundation's intellectual merit and broader impacts review criteria.- subawards are not planned for this award.